Learn Authelia - Monitoring & Logging

#Introduction

Episode 25 deployed Authelia to Kubernetes with HPA, probes, and network policies. The infrastructure is running, but there's one question still unanswered: how do you know Authelia is actually healthy? A flight doesn't rely on the pilot's feelings — it relies on instruments. Episode 26 builds those instruments: structured logs, metrics, dashboards, and alerting.

Why is observability for Authelia so important? Because Authelia is the gate. A small problem at the gate — server down, broken session storage, a brute force surge — impacts every application behind it. The earlier it's detected, the smaller the blast radius. Monitoring here isn't an add-on; it's the safety net for the entire architecture built since episode 13.

#Structured Logging

Authelia writes logs to stdout (standard for containers) in two formats: text and json. For production, use JSON — a structured format that can be parsed by machines, filtered, and aggregated without ambiguity. Enable it via the log block in the configuration:

log:
  level: info
  format: json
  file_path: ''

file_path is left empty so logs keep flowing to stdout and are handled by the runtime (Docker logging driver, kubelet, etc.). Don't write logs to a file inside the container — that file disappears when the container is replaced, and it isn't rotated automatically.

An example JSON log line produced when a request is processed:

{"time":"2026-08-03T10:00:00Z","level":"info","msg":"Request processed","method":"GET","path":"/api/verify","status_code":200,"latency_ms":12}

With a format like this, you can make queries like "all requests to /api/verify that returned 5xx in the last 5 minutes" — something almost impossible to do with plain text logs.

#The Right Log Level

Authelia log levels: trace, debug, info, warn, error. A rule of thumb:

• Production: info — balanced between information and volume.
• Incidents / investigations: debug — reveals authorization decisions and session flows.
• Only when requested by a vendor/maintainer: trace — extremely noisy and may include sensitive data.

You can change the level without restarting via hot reload, but remember: permanent debug and trace in production is a performance leak and a privacy risk — a topic you already learned in episode 23.

#Metrics: The Prometheus Endpoint

Authelia ships with a built-in Prometheus exporter. Enable it via the telemetry block — the endpoint stands on a separate port (default :9959) so it isn't mixed with authentication traffic:

telemetry:
  metrics:
    enabled: true
    address: 'tcp://:9959/metrics'

Never expose port 9959 to the public. Metrics don't contain credentials, but they provide a complete map of authentication patterns — valuable enough for an attacker during reconnaissance.

#Meaningful Metrics

Authelia exposes counters and histograms prefixed with authelia_. The most important metrics to monitor:

From these you can compute the authentication success-failure ratio, detect banned=true spikes (a sign of brute force held back by regulation), and measure authentication latency. Note that metric names can change between versions — when moving versions, compare the /metrics output before updating dashboards.

#Scraping with Prometheus

Add a scraping job in prometheus.yml. If Authelia runs in Docker Compose, just target the service name; in Kubernetes, use a ServiceMonitor:

scrape_configs:
  - job_name: authelia
    metrics_path: /metrics
    scheme: http
    static_configs:
      - targets: ['authelia:9959']

With this job, Prometheus pulls metrics every 15 seconds and stores them in a time series — the raw material for dashboards and alerts.

#Grafana Dashboard

Authelia provides a community dashboard that can be imported directly into Grafana as a starting point. That dashboard shows request summaries, authentication failure rates, and latency — complete with panels using the metrics above.

Don't stop at the built-in dashboard. Build custom panels that answer your own operational questions:

• How long was the average authentication today compared to yesterday? (the authelia_authn_duration histogram)
• How many active sessions are there? (from logs or Redis metrics)
• How many requests were banned by regulation? (sum authelia_authn_total with banned="true")

A good dashboard turns numbers into decisions, not just a collection of pretty panels.

#Alerting: Alarms Before It's Too Late

A dashboard is only useful while you're watching it. Alerting works 24 hours. Define rules in Prometheus — here's an example for the three most important scenarios:

groups:
  - name: authelia
    rules:
      - alert: AutheliaDown
        expr: up{job="authelia"} == 0
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "Authelia tidak merespons scraping"
 
      - alert: AuthenticationFailuresSpike
        expr: rate(authelia_authn_total{success="false"}[5m]) > 10
        for: 2m
        labels:
          severity: warning
        annotations:
          summary: "Lonjakan kegagalan autentikasi 1FA"
 
      - alert: HighServerErrors
        expr: sum(rate(authelia_request_total{code=~"5.."}[5m])) > 0.05
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "Terlalu banyak respons 5xx dari Authelia"

Forward the alerts to a channel the team actually watches: Telegram, Discord, Slack, or email — via Alertmanager. The two alarms that most often save production are the spike in failed authentications (a sign of attack or broken config) and AutheliaDown (a sign the whole gate is closed).

#Log Aggregation with Loki

Metrics tell you whether there's a problem; logs tell you why. During an incident, you need to read logs from all instances at once, not open terminals one by one. Loki — Prometheus's natural pair from Grafana — accepts Authelia's JSON logs and indexes them by label:

discovery.relabel "authelia" {
  targets = discovery.docker.targets("docker", {}).output
  rule {
    source_labels = ["__meta_docker_container_name"]
    regex = "authelia-.*"
    action = "keep"
  }
  rule {
    source_labels = ["__meta_docker_container_name"]
    target_label = "container"
  }
}

With the container="authelia" label, a Loki query like {container=~"authelia.*"} |= "level\":\"error" immediately shows all errors from all instances in one place. Promtail for file-based setups or kube-prometheus-stack for Kubernetes are common alternatives.

#Closing

Episode 26 completes the observability layer: enabling structured JSON logs at the right level, turning on the Prometheus metrics endpoint on port 9959, scraping it with a job, displaying it in Grafana dashboards, enforcing alerting for critical scenarios, and aggregating logs centrally with Loki.

Key points:

• Use log.format: json and let logs flow to stdout.
• Metrics live at telemetry.metrics.address with the /metrics path — without the path, the endpoint 404s.
• Monitor authelia_authn_total and authelia_authn_second_factor_total for attack patterns.
• Alert for AutheliaDown, authentication failure spikes, and 5xx.
• Loki answers "why" once metrics have answered "whether".

All the data is collected — now the question: what happens if everything is lost? In episode 27 we dissect Backup & Disaster Recovery: what must be backed up, strategies, restore procedures, and disaster scenarios. See you in episode 27!