Learn 9router - Core Concepts & Main Architecture

#Introduction

In episode 1 you understood why the AI routing gateway is needed and the use cases that motivate it. Now it's time to tear open the engine: how 9router works from an incoming request to the returned response. This is the most fundamental episode in the early phase of the series — every later episode will reference the concepts you learn here.

This episode's roadmap: we'll look at the big picture of the architecture, dissect the four core components (route engine, model selector, policy engine, observability layer), trace the flow from intent extraction to tool invocation, and finally map out 9router's position between clients, LLM providers, and downstream tools.

#9router Architecture Overview

Conceptually, 9router is a chain of components working like a factory: requests enter at one end, decisions are made in the middle, and model calls happen at the other end. Four core components form that factory:

Klien/App --> [Route Engine] --> [Policy Engine] --> [Model Selector]
                 ^                    ^                    |
                 |                    |                    v
            [Intent Extraction]   [Observability]    LLM Provider / Tools

The four core components are:

#Route Engine: The Heart of Decision-Making

The route engine is the component that receives the request first and runs the routing decision pipeline. It extracts information from the request (path, headers, metadata, and prompt content), evaluates matching rules, and determines which route is selected.

Think of the route engine as an airport information officer: it receives the passenger (request), asks for the destination (intent), checks identity (user identity), then directs them to the correct gate (route). This decision isn't a single step, but a chain of evaluations you'll see in episode 4.

#Intent Extraction: Understanding Request Meaning

Before a routing decision is made, 9router needs to understand the intent behind the request. Intent extraction turns natural language prompts into labels that can be evaluated deterministically. For example, the prompt "summarize this email" is extracted into the intent summarization, while "create a picture of a cat" becomes image-generation.

Two approaches are used in intent extraction:

The keyword approach suits requests with clear patterns, while semantic classification handles a wide range of language variation. 9router allows combining both in a single pipeline, with the semantic classifier as a complement when keywords fail.

#Model Selector: Choosing the Right Model

After the route is determined, the model selector picks the most suitable target model. This decision weighs criteria like performance, cost, accuracy, and provider availability. The model selector understands the registered stack of models — including conversation, embedding, code, and vision models — then picks the one that best matches the task type and applicable policy.

As an example, a route might choose between three models for the summarization task: a cheap, fast model for short documents, a premium model for complex documents, and a fallback when the primary provider is down. The model selector evaluates these criteria and produces a decision that can be audited.

#Policy Engine: Business and Security Rules

The policy engine is the layer that ensures every routing decision complies with business and security policy. It evaluates things like: is this user allowed to use a premium model? Does this request's region permit that provider? Has the monthly quota been reached? Does the prompt content fall into a prohibited category?

The policy engine works before the model is called. If a request violates policy, it is rejected or redirected to a fallback — not forwarded to the model. This is what makes safety applicable consistently in one layer, rather than in each application. Details on enforcement, rate limiting, and blocked intents will be covered in depth in episode 6.

#Observability Layer: The Gateway's Eyes and Ears

Every request passing through the gateway leaves a trace. The observability layer captures three kinds of data: metrics (latency, request volume, success rate per route), logs (request context, selected route, policy evaluation results), and traces (the request's journey from client to provider). This data is the primary material for troubleshooting and optimizing routing in episode 7.

#Tool Invocation: Going Beyond Model Calls

9router doesn't just forward requests to LLMs; it can also trigger tool invocation — calling an external tool or service as part of an agentic flow. For example: after the model produces an answer, a step calls a search API to fetch additional data, then the result is sent back to the model for assembling the final answer.

Tool invocation is the point where the gateway crosses the boundary of "traffic forwarder" and becomes an agentic orchestrator. This is what enables more complex workflows like retrieval-augmented generation and multi-step function calling. To follow each step's trail directly, use the 9router logs --follow command while debugging agentic flows.

#9router's Position Between Clients, Providers, and Tools

Now we can map out 9router's position in the complete architecture. It stands at three junctions:

• Client to LLM provider: applications send one request to 9router, not to many different providers.
• Policy to model call: every request passes through the policy engine before touching a model.
• Model to tools: when a model needs external data, 9router manages the calls to tools and downstream API services.

Here's a simple illustration:

   [Mobile/Web App] --> [9router] --> [OpenAI]
                              |--> [Anthropic]
                              |--> [Azure OpenAI]
                              `--> [Search API / Tools]

The consequences of this design are significant: applications only know one endpoint and one API format. Provider swaps, model changes, or new policies never touch application code at all — just change the configuration at the gateway.

#The Complete Flow of One Request

Let's put it all together in one real flow. When an application sends the request "summarize this document" to 9router:

1. The route engine receives the request and extracts user metadata and payload.
2. Intent extraction classifies the prompt as intent summarization.
3. The policy engine checks access permissions, region, and user quota.
4. The model selector picks a summarization model that matches the user's tier.
5. The request is forwarded to the chosen LLM provider.
6. The observability layer records the route, model, latency, and policy evaluation results.
7. The response is returned to the application complete with routing metadata.

This flow happens in milliseconds and leaves an auditable trail at any time. If any step fails, fallback and circuit breaker take over — topics we'll discuss in later phases.

#Conclusion

In episode 2 you've dissected 9router's internal architecture: the four core components working together, the flow from intent extraction to tool invocation, and the gateway's position between clients, providers, and tools. This understanding is the foundation for all the hands-on episodes that follow.

Key takeaways:

• The route engine runs the decision pipeline; intent extraction turns prompts into evaluable labels.
• The model selector chooses models based on performance, cost, accuracy, and availability.
• The policy engine ensures every decision complies with business and security rules before the model is called.
• The observability layer captures metrics, logs, and traces for every routing decision.
• 9router stands between clients, LLM providers, and tools — applications only know one endpoint.

In the next episode, episode 3, we'll install 9router and do the basic setup — create a workspace, configure the first route rules and model endpoints, then verify real request and response routing. Get your local emulator ready, because we start hands-on practice!