Learn Traefik - Metrics & Prometheus

#Introduction

Traefik without observability is a black box: requests drift away, errors appear from who knows where. Episode 23 turns on the lights with metrics. Traefik exposes Prometheus metrics on the /metrics endpoint — structured data about every entrypoint, router, and service that fuels dashboards and alerting.

We will enable metrics, learn the available metric types, select the most important metrics, connect Prometheus for scraping, then display them in Grafana with PromQL. By the end of the episode, you can answer the question "what percentage of our requests fail?" in seconds.

#Setting Up Prometheus Metrics

#The /metrics Endpoint

Enable the metrics provider in static config:

metrics:
  prometheus:
    addEntryPointsLabels: true
    addServicesLabels: true
    entryPoint: metrics
    buckets:
      - 0.1
      - 0.3
      - 1.2
      - 5.0

• addEntryPointsLabels: true: metrics are labeled with the entrypoint name.
• addServicesLabels: true: metrics are labeled with the service name.
• entryPoint: metrics: the /metrics endpoint is published on an entrypoint named metrics.
• buckets: histogram boundaries for latency in seconds.

The metrics entrypoint must be defined and isolated — do not expose it to the public:

entryPoints:
  metrics:
    address: ":9100"

Other supported metrics providers: Datadog, StatsD, and InfluxDB — similar configuration patterns, adapted to each format.

#Scrape Configuration

#Prometheus Pulls the Data

Prometheus is directed to scrape Traefik's /metrics endpoint. The job configuration in prometheus.yml:

scrape_configs:
  - job_name: "traefik"
    static_configs:
      - targets: ["traefik:9100"]
        labels:
          instance: "edge"

With this setup Prometheus pulls metrics every default scrape interval (15 seconds) and stores them with the instance: edge label. Confirm from the Traefik side:

curl -s http://localhost:9100/metrics | head -n 20

The curl command above shows metric lines in Prometheus text exposition format — the first key metric is traefik_entrypoint_requests_total.

#Important Metrics

#Metric Types

Prometheus has three basic types that all Traefik metrics use:

• Counter: a value that only increases — e.g. traefik_service_requests_total.
• Gauge: a value that can go up and down — e.g. the number of open connections.
• Histogram: a distribution of values — e.g. request latency.

#Metrics You Must Know

traefik_entrypoint_requests_total        -> total requests per entrypoint
traefik_entrypoint_request_duration_seconds -> latency per entrypoint
traefik_entrypoint_requests_bytes_total  -> request size
traefik_entrypoint_responses_bytes_total -> response size
traefik_router_requests_total            -> total requests per router
traefik_router_requests_tls_total        -> TLS requests per router
traefik_service_requests_total           -> total requests per service
traefik_service_request_duration_seconds -> latency per service
traefik_service_open_connections         -> open connections per service
traefik_service_requests_bytes_total     -> request size per service

These metrics have labels such as code, method, entrypoint, router, and service — the combination of labels is what gives PromQL queries their flexibility.

#Grafana and PromQL

#Queries for Alerting

Some of the most useful PromQL queries:

sum(rate(traefik_service_requests_total{code=~"5.."}[5m]))
  / sum(rate(traefik_service_requests_total[5m]))

The query above calculates the ratio of 5xx requests to all requests in 5 minutes — the key health metric of a service.

histogram_quantile(0.95,
  sum(rate(traefik_service_request_duration_seconds_bucket[5m])) by (le))

This shows the 95th percentile latency — if it spikes, a backend is slowing down.

#Grafana Dashboards

Grafana displays those queries visually. Traefik Labs provides an official dashboard that can be imported directly into Grafana (search for "Traefik" in grafana.com dashboards). That dashboard shows the entrypoint map, error rate, latency, and service status all at once — an excellent starting point before you create custom dashboards.

#Closing

Key takeaways:

• Enable metrics.prometheus and publish it on a dedicated /metrics entrypoint.
• Metric types: counter, gauge, and histogram.
• Important metrics are grouped per entrypoint, router, and service.
• Prometheus scrapes with a simple job in prometheus.yml.
• PromQL rate and histogram_quantile for error ratio and latency.
• The official Traefik Grafana dashboard is the best starting point.

In episode 24 next we will cover access logs & debugging — enabling access logs with Common and JSON formats, custom filters and fields, centralized log aggregation with Loki and ELK, and debugging techniques with log levels, API inspection, and the dashboard.