Learn 9router - Model & Tool Selection

#Introduction

In episode 4 you learned where requests are directed. Now we focus on which model is chosen and which tool gets called. Model selection is the heart of cost and quality optimization for an AI gateway — small decisions here have a big impact on the monthly bill and the user experience.

This episode's roadmap: we'll discuss model selection criteria (performance, cost, accuracy), build a multi-model stack for four task categories (conversational, embeddings, code, vision), and finally integrate external tools and API services into the routing flow. By the end of this episode you'll be able to design a balanced model selection strategy.

#Model Selection Criteria: Performance, Cost, Accuracy

No single model is perfect for every task. Model selection in 9router balances three main criteria:

These three criteria clash with each other: the most accurate model is usually the slowest and most expensive. That's why 9router evaluates a model's fitness based on each task's needs, not a global ranking. A small model can be the best choice for light tasks even if its accuracy ranking is lower.

To make this decision easier, declare model profiles with complete metrics:

models:
  - id: gpt-4o-mini
    provider: openai
    credential: OPENAI_API_KEY
    metrics:
      latency_p95_ms: 350
      cost_per_1k: 0.00015
      accuracy: 0.82
  - id: gpt-4o
    provider: openai
    credential: OPENAI_API_KEY
    metrics:
      latency_p95_ms: 1200
      cost_per_1k: 0.0025
      accuracy: 0.93

The metrics above become the evaluation material when 9router decides which model best fits a route. The more accurate the metric data — ideally from the observability in episode 7 — the better the resulting decisions.

#Model Selection Strategies in a Route

There are two approaches to determining the target model within a route: explicit and evaluation-based. The explicit approach points to a specific model, as you saw in episode 3. The evaluation approach lets 9router choose among several candidates based on metrics:

routes:
  - name: summarization
    match:
      intent: summarization
    target:
      models:
        - gpt-4o-mini
        - gpt-4o
      select: best-score
      weight:
        cost: 0.6
        accuracy: 0.3
        latency: 0.1

With the configuration above, 9router computes a weighted score for each candidate and picks the best one. The weighting can be tuned per route: a paid-service route can weight accuracy high, while a free-feature route weights cost high. This is real policy-based model selection.

#Multi-Model Stack: Conversational, Embeddings, Code, Vision

Modern AI applications use more than one model category. 9router supports a diverse stack under a single entry point — here's the category map:

Each category is registered as a separate model endpoint, then different routes handle them based on task type:

routes:
  - name: chat
    match:
      taskType: conversation
    target:
      model: gpt-4o
  - name: embed
    match:
      taskType: embedding
    target:
      model: text-embedding-3-small
  - name: codegen
    match:
      taskType: code
    target:
      model: claude-code
  - name: vision
    match:
      taskType: vision
    target:
      model: gpt-4o-vision

Applications just send a request with the task type field, and 9router directs it to the right category. Applications don't need to know which provider handles which task — one endpoint, many models behind it.

#Integration with External Tools and API Services

Finally, models aren't the only routing targets. In agentic flows, requests often need to call external tools — search APIs, databases, or internal services. 9router manages these tools as routing targets just like models:

tools:
  - id: search-api
    kind: http
    url: https://api.example.com/search
    method: POST
    credential: SEARCH_API_KEY
    timeout_ms: 2000
  - id: weather-api
    kind: http
    url: https://api.example.com/weather
    method: GET
    credential: WEATHER_API_KEY

Then a route triggers the tool when a specific intent is detected:

routes:
  - name: weather-intent
    match:
      intent: weather
    target:
      tool: weather-api
    chain:
      - next: final-chat
        on: tool_result
  - name: final-chat
    match:
      any: true
    target:
      model: gpt-4o

The flow above illustrates a real pattern: the prompt "what's the weather like tomorrow?" is classified as intent weather, triggers a weather tool call, then the result is sent to the final model to be assembled into an answer. This pattern uses route chaining from episode 4 to build a complete agentic pipeline.

#Practice: Testing Model & Tool Selection

Verify your selection strategy by sending a request for each task type, then check the selected model via routing metadata:

curl -X POST http://localhost:8080/v1/chat \
  -H "Authorization: Bearer <API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{"prompt":"tulis fungsi validasi email","taskType":"code","include_route_meta":true}'

Notice the model field in the response — it should show claude-code from the codegen route. Repeat with other taskType values and compare the results. If a request triggers a tool, the metadata will also show the tool call details along with its latency.

To test fallback behavior when the primary model is down, temporarily disable the primary model's credentials and send the same request. The gateway should try the fallback models in declaration order — an important verification for the resilience we'll deepen in episode 18. Also make sure the whole configuration stays valid by running 9router validate before each test.

#Conclusion

In episode 5 you've understood the art of model and tool selection: the three selection criteria (performance, cost, accuracy), explicit and metric-evaluation strategies, building a multi-model stack for four task categories, and integrating external tools into the agentic routing flow.

Key takeaways:

• Model selection balances performance, cost, and accuracy — not choosing the absolute best model.
• Per-route metric weighting enables different strategies for paid services and free features.
• Separate conversation, embedding, code, and vision task types into different routes for per-category cost optimization.
• External tools are registered like models and triggered by intent through route chaining.
• Test selection via routing metadata and test fallback by disabling the primary model's credentials.

In the next episode, episode 6, we'll discuss policy enforcement and safety — policy-driven routing for compliance, privacy, and access control, rate limiting and quota enforcement, up to safe fallback routes and blocked intent handling. We'll complete your selection strategy with the security layer that production requires!