Learn 9router - Distributed & Multi-region Routing

#Introduction

Episode 16 made the gateway smart with code — hooks, plugins, actions, and route modules. But that intelligence still lives in one geographic region. When users in Jakarta, Berlin, and São Paulo use the same service, a gateway in a single region forces one of them to travel a very long distance — and the latency from episode 15 comes back, this time in the form of physical distance.

Episode 17 takes 9router to world scale: edge routing that answers from the nearest location, regional model selection that leverages the nearest provider, strategies for reducing latency for global users, and multi-region failover plus provider redundancy so the gateway stays alive even when one region or one provider falls.

#Edge Routing: Answering from the Nearest Location

The first principle: be close to the user. A single gateway in a single region forces all requests across intercontinental networks. By placing 9router instances in several regions — or at the edge — incoming requests are handled by the nearest instance based on client location, minimizing travel distance.

Defining gateway regions and endpoints

regions:
  - name: ap-southeast
    endpoints: [gw-id, gw-sg]
    primary: true
  - name: eu-central
    endpoints: [gw-fra]
  - name: us-east
    endpoints: [gw-nyc]

Endpoint selection can be based on DNS with geo-routing or be latency-aware: instances monitor RTT distance between endpoints and direct requests to the fastest one at that moment. Every instance shares the same configuration, so route decisions stay consistent no matter where the request enters.

Viewing the nearest endpoint for a region

9router edge whoami
9router edge routes --from ap-southeast

#Regional Model Selection

Proximity to the user is only half the journey; LLM providers also have locations. Requests from Europe shouldn't round-trip to US data centers if there's a nearer model region. 9router maps models to provider regions and picks the best candidate based on request location.

Mapping models per provider region

model_regions:
  - model: gpt-4o-mini
    regions:
      - provider: azure-asia
        region: southeast-asia
      - provider: azure-eu
        region: france-central
  - model: claude-sonnet
    regions:
      - provider: anthropic-eu
        region: eu-central

When a request comes from Jakarta, 9router ranks azure-asia first because it's close; a request from Berlin picks azure-eu. This ordering is called the preference list — it can still be shifted by cost policy, SLA, or the data residency from episode 14.

Combining regions with residency policy

model_regions:
  - model: gpt-4o-mini
    regions:
      - provider: azure-eu
        region: france-central
    only_if:
      residency: eu

#Reducing Latency for Global Users

Edge routing and regional providers handle distance, but there's another layer: the parts of a request that don't need to travel far at all. Caches — from episode 15 — placed at the edge can serve answers to popular questions directly from the nearest location without touching origin or provider.

Cache spread across the edge

edge_cache:
  enabled: true
  response_ttl: 300s
  semantic_cache:
    store: redis-cluster
    topology: geo-distributed
  warm_on: [popular_intents]

Cache warming is the technique of pre-fetching: intents known to be popular are pulled from origin to other regions' caches in advance, so the first user in a new region doesn't wait the full time. The combination of geo-distributed Redis and warming keeps p95 latency low even when users are spread across many continents.

Success

For requests that really must reach the model, use streaming: users see the first token far faster than waiting for the complete answer. The TTFT from episode 15 becomes the biggest winner at global scale.

#Multi-region Failover

When one region is disrupted, users shouldn't just "wait until it recovers". Multi-region failover directs requests to another healthy region — transparent to clients who don't know exactly where their request is served.

Failover between gateway regions

regions:
  - name: ap-southeast
    endpoints: [gw-id, gw-sg]
    failover:
      - eu-central
      - us-east
    health_check:
      interval: 5s
      timeout: 2s

The instance in ap-southeast monitors its own health. If both endpoints die at once or a health check fails, traffic is diverted to eu-central then us-east. The client keeps using one domain; the failover logic lives in configuration, not the application.

Manually shifting traffic and checking health

9router failover trigger --from ap-southeast --to eu-central
9router region status

Manual failover is useful for planned migrations — for example data center maintenance — while automatic failover handles the unexpected. Combine both and make sure switchback is also tested: returning to the primary region once it has recovered, rather than letting traffic stay on the backup region forever.

#Provider Redundancy

A single provider isn't a guarantee. If openai-prod is disrupted, every request depending on it goes down too. Provider redundancy means each model has more than one equivalent provider — and 9router chooses among them based on health and preference.

One model, many providers

routes:
  - name: chat-general
    match:
      intent: general_chat
    model: gpt-4o-mini
    providers:
      - openai-prod
      - azure-openai-prod
      - openrouter-fallback
    preference: latency

The providers order is a preference list; preference: latency makes 9router weigh latency when choosing, not just the order. If the first provider refuses, the gateway moves to the next automatically — a pattern you've seen since episode 3, now multiplied across all regions.

Warning

Provider redundancy requires model consistency between providers. Test the output of "equivalent" models periodically — two providers claiming the same model name can behave differently.

#Configuration Synchronization and Consistency

Distribution raises a crucial question: how do all regions know the same policy? If config is sent manually to each region, one region can lag behind and make decisions with old rules. 9router uses a config center as the single source of truth, distributed automatically.

Distributing configuration to all regions

config_sync:
  source: git
  registry: control-plane
  targets: [gw-id, gw-sg, gw-fra, gw-nyc]
  versioned: true
  rollback_on_error: true

Config is managed in git (exactly the pattern from episode 10), registered to the control plane, then distributed to all regions with the same version. rollback_on_error automatically reverts to the last healthy version if one region fails to apply — while the config_sync audit records every change for episode 14 purposes. Distribution status can be monitored with 9router config sync status.

#Conclusion

Episode 17 takes the gateway to world scale: edge routing answers from the nearest location, regional model selection leverages providers on the same continent, edge caching and streaming cut latency for global users, multi-region failover keeps the service up when one region falls, provider redundancy counters dependence on a single vendor, and configuration synchronization ensures all regions speak with the same rules.

Key takeaways:

Bring the gateway close to users with multi-region or edge, and select endpoints by latency.
Map models to provider regions so requests are served from the nearest location, aligned with residency policy.
Put caches at the edge and warm popular intents; streaming keeps TTFT low globally.
Set up tiered failover between regions and test switchback regularly, not just the path to the backup.
Provider redundancy and versioned config sync make the whole system consistent and disaster-tolerant.

Your gateway now serves the whole world. In episode 18 we face the toughest moments: Recovery & Operational Resilience — failover strategies for provider outages, backup routes and degraded mode behavior, and incident response for routing failures. See you there!