Learn 9router - Observability Basics

#Introduction

Episode 6 closed the security side: you can now block dangerous requests, enforce quotas, and prepare safe fallbacks. But a secure gateway without visibility is like an alarm system that's never checked — the rules might be working, but you'd never know.

Episode 7 builds eyes for your gateway. We'll cover 9router's basic observability from three sides: metrics for model latency, request volume, and success rate; logging to record request context, the selected route, and provider diagnostics; and tracing to follow the AI request flow from client to provider. You'll understand how to measure routing decisions, not just look at them.

#The Three Pillars of Observability on a Routing Gateway

Observability on an AI gateway is slightly different from an ordinary service. There's an extra layer: the routing decision itself is a primary diagnosis input. If a request is slow, the cause could be the provider, the route policy, or the model selector picking wrong. That's why the collected data must retain the decision trail, not just the numbers.

The concept of the three pillars: metrics are periodic numeric aggregates (how many requests, how long, how many succeeded), logs are discrete contextual events (route X was chosen because Y), and tracing is the connected flow across components (from client, to gateway, to provider, and back). All three complement each other: metrics show the symptoms, logs give the details, tracing shows the chain of causes.

#Metrics: Latency, Volume, and Success Rate

9router exposes standard OpenTelemetry metrics that Prometheus can scrape. The three mandatory metrics to start: latency per model, request volume, and success rate. Add labels like route name, provider, and status so they can be broken down.

metrics:
  enabled: true
  endpoint: /metrics
  prefix: nine
  labels: [route, provider, model, status]
  histograms:
    - name: latency
      buckets_ms: [50, 100, 250, 500, 1000, 2500, 5000]

From the /metrics endpoint you get numbers like this:

{
  "nine_requests_total": { "value": 48213, "labels": { "route": "chat-primary", "provider": "openai", "status": "success" } },
  "nine_latency_seconds": { "value": 1.24, "labels": { "model": "gpt-4o-mini" } },
  "nine_success_rate": { "value": 0.986, "labels": { "route": "chat-primary" } }
}

With the route and provider labels, you can directly answer questions like "which route is the slowest?" or "which provider is starting to fail?" without guessing. For periodic collection, add a scrape job to Prometheus or install the 9router exporter: 9router metrics export --otel-endpoint http://localhost:4318.

#Connecting to a Dashboard

Raw metrics are useless without visualization. A scrape job in Prometheus reads the /metrics endpoint periodically, and Grafana turns those numbers into dashboards. One mandatory panel: p95 latency per route broken down by provider, stacked with success rate and request volume so volume spikes correlate visibly with speed drops.

scrape_configs:
  - job_name: nine-gateway
    metrics_path: /metrics
    static_configs:
      - targets: [nine-gateway:8080]

Don't forget to add basic alerts on top of these metrics — for example success rate below a threshold or a p95 latency spike. Metrics that are collected but never looked at are no better than having no metrics at all.

#Logging Request Context and Route Decisions

9router logs are structured JSON — not free text — so they can be filtered and aggregated. Every log entry carries complete request context: request ID, selected route, intent score, target provider, duration, and policy results. This is what distinguishes AI gateway logging: routing decisions are recorded as data.

{
  "ts": "2026-08-03T10:21:07Z",
  "request_id": "req_8f3kq2",
  "tenant": "acme-corp",
  "intent": "chat",
  "confidence": 0.94,
  "route": "chat-primary",
  "policy": ["privacy-first", "api-guard"],
  "target": "openai/gpt-4o-mini",
  "status": "success",
  "duration_ms": 812
}

Notice the policy field: this log records which policies were evaluated. When a request suddenly changes direction, you can trace the "why" through logs, not just "what happened". To read logs in real time use 9router logs --follow --filter route=chat-primary, and for a specific time range 9router logs --since 2h --query status=error.

#Tracing AI Request Flows End-to-End

Tracing connects every stage of one request: client, gateway, policy evaluation, model selector, provider, tool invocation, up to the response. In 9router, every stage is a span within one trace, all chained together via the request ID and the traceparent header.

tracing:
  enabled: true
  exporter: otlp
  endpoint: http://collector:4317
  service_name: nine-gateway
  span_attributes: [route, intent, model, provider]

With an OTLP collector, traces can be sent to Jaeger, Tempo, or any observability platform. The interesting part of AI tracing: spans for provider calls can include attributes like input and output token counts, the model used, and provider queue wait time. This enables breakdowns like "of 2 seconds total, 1.2 seconds was actually in the provider call".

For requests crossing components, the trace context can be propagated from the client: send the traceparent header to the gateway and 9router will join it to internal spans so the entire flow appears as one complete trace.

#Conclusion

9router's basic observability lets you see routing decisions as data: metrics give aggregate numbers per route, provider, and model; structured logs record request context along with policy decisions; tracing chains all stages in one end-to-end flow. With these three pillars, incidents stop being riddles.

Key takeaways:

• The mandatory first metrics: model latency, request volume, and success rate, with route and provider labels.
• Structured JSON logs must include request ID, intent, selected route, and evaluated policies.
• Never write raw PII conversation payloads into logs.
• Cross-component tracing uses OpenTelemetry with OTLP, and span attributes carry AI context like token usage.
• Forward the traceparent header from the client so the entire request flow appears as one trace.

In episode 8 we start playing with bolder patterns: Advanced Routing Patterns — traffic splitting, canary routes, shadowing, A/B testing, up to dynamic routing based on runtime signals. See you there!