Learn 9router - Multi-tenant & Customer-specific Routing

#Introduction

In episode 8 you played with traffic splitting, canary, shadowing, A/B testing, and dynamic routing. All those patterns assume a single traffic owner. Now we add the dimension that takes an AI gateway into the enterprise realm: many customers on one gateway.

Episode 9 covers multi-tenant and customer-specific routing. We'll break down tenant-aware route rules, per-customer model preferences, isolation of sensitive workloads with different SLAs, and custom routing based on user profiles and domains. By the end of this episode you'll be able to build a gateway that treats each customer like its own private gateway — without deploying separate gateways.

#The Concept of Multi-tenancy in 9router

Multi-tenancy means one gateway serving many tenants with different rules. In 9router, each request is attributed to an owner via context — usually headers or claims from the auth token. This context flows through the entire pipeline: policy evaluation, route matching, up to model selection, exactly as you started learning in episode 4.

tenants:
  - id: acme-corp
    domain: acme.example.com
    plan: enterprise
  - id: startup-labs
    domain: labs.example.dev
    plan: starter

When a request arrives, 9router matches its context to one of the tenants. The routing decision can then depend on that tenant. This differs from a mere label: a tenant can influence which policies are active, which models may be used, and which quotas apply.

#Tenant-aware Route Rules

Tenant-aware route rules let two customers sending similar questions get different treatment. For example, an enterprise tenant wants the strongest model for all requests, while a starter tenant is limited to lightweight models.

routes:
  - name: chat-enterprise
    match:
      tenant: acme-corp
      intent: chat
    target: gpt-4o
  - name: chat-starter
    match:
      tenant: startup-labs
      intent: chat
    target: gpt-4o-mini

Routes are still evaluated top to bottom. The chat-enterprise route is more specific because it names a tenant, so requests from acme-corp are satisfied there and never fall to the starter route. These rules also allow forcing a specific tenant through a specific regional provider instance for compliance. The list of active tenants can be checked with 9router tenants list.

#Per-customer Model Preferences

Many customers have their own model preferences — for example a contract requiring Anthropic, or the need for embeddings and chat to use the same provider. 9router stores these preferences as part of the tenant profile and applies them when the model selector works.

tenants:
  - id: acme-corp
    preferences:
      chat_model: anthropic/claude-3-5-sonnet
      embedding_model: openai/text-embedding-3-large
      fallback_models: [openai/gpt-4o-mini]

The final model decision can still be overridden by other constraints — for example the compliance policy from episode 6 or the dynamic signals from episode 8. Preferences are a default, not a mandate. Make sure each override is logged as context, so you know when a preference was overridden and why.

#Sensitive Workload Isolation and Different SLAs

Not all tenants are equally important. Tenants with sensitive workloads — health data, financial data, or children's data — need stricter isolation: dedicated providers, dedicated regions, and configuration that never shares context with other tenants. On the other hand, enterprise tenants need higher SLAs than starter tenants.

tenants:
  - id: health-plus
    plan: enterprise
    isolation:
      mode: dedicated
      provider_region: ap-southeast-1
      context_isolated: true
    sla:
      p95_latency_ms: 1500
      availability: 0.995
  - id: startup-labs
    plan: starter
    isolation:
      mode: shared
    sla:
      p95_latency_ms: 4000
      availability: 0.95

Isolation in the form of context_isolated ensures state and history never leak between tenants, while mode: dedicated guarantees provider capacity isn't contended with other tenants. The different SLAs are then reflected in quotas, queue priority, and observability targets — enterprise tenants get the fast lane when the provider is full.

#Custom Routing Based on User Profiles and Domains

The most granular layer: routing decided from a user's profile or business domain, not just the tenant. For example, admin users get analytics tool access, support users are directed to cost-efficient models, and requests from a specific domain are directed to a specific tool pipeline.

routes:
  - name: support-chat
    match:
      tenant: acme-corp
      role: support
    target: gpt-4o-mini
  - name: admin-tools
    match:
      tenant: acme-corp
      role: admin
      tool_scope: analytics
    target: gpt-4o
    tools: [analytics-query, report-generator]

With role and tool_scope attributes taken from the auth token, one tenant can have a different routing experience per role. This keeps costs in check: roles that need expensive models get them, roles that don't are directed to economical models, and tool access is limited per user profile.

#Conclusion

Multi-tenant routing turns one gateway into many virtual gateways. Tenant context marks the ownership of each request, tenant-aware rules separate treatment between customers, model preferences become defaults that can be overridden with recorded reasons, isolation protects sensitive workloads, and different SLAs ensure premium customers are prioritized. User profiles and domains add precision down to the role level.

Key takeaways:

• Tenant context flows from authentication through the entire pipeline and is always recorded in logs and metrics.
• Route rules can be matched against tenants to apply different treatment within one gateway.
• Per-customer model preferences are defaults; every override must be recorded with its reason.
• Context isolation between tenants is a security boundary that must be explicitly tested.
• Role- and tool-scope-based routing keeps costs in check by giving the right model to the right role.

In episode 10 we shift from designing to managing: Configuration Management & Versioning — route config as code, validation and dry runs, safe rollback, up to configuration evolution for long-lived agent workflows. See you there!