Learn RabbitMQ - Monitoring & Observability

#Introduction

So far we've checked RabbitMQ manually: rabbitmqctl status, looking at logs, or opening the Management UI. That all works for learning, but in production you need automatic, continuous supervision. A broker with problems must be visible within seconds, not after customers complain.

This episode builds a complete observability layer. First, we get to know the metric sources: the Management UI, the HTTP API, and rabbitmqctl commands. Second, we integrate Prometheus — the industry standard for metrics — through the official plugin, and display it in Grafana. Finally, we set up proper logging: levels, locations, structured logging, and aggregation to a centralized system.

With good observability, you no longer need to guess why queues are piling up — the data is available, and the alarms await in episode 25.

#Monitoring Approaches

#Management UI and HTTP API

The Management UI shows real-time metrics per node, vhost, queue, and exchange. Behind it is the HTTP API, which can be called programmatically:

curl -s -u arman:pass \
  http://localhost:15672/api/queues/%2f/task_queue | python3 -m json.tool

The curl command above fetches metrics for the task_queue queue in the / vhost (%2f). The JSON response contains the message count, publish rate, deliver rate, and more.

#rabbitmqadmin and rabbitmqctl

rabbitmqadmin provides a convenient interface to the HTTP API, while rabbitmqctl reads internal state directly:

rabbitmqctl list_queues name messages messages_unacknowledged consumers

The rabbitmqctl list_queues command shows depth, unacknowledged messages, and consumer count — three key numbers for assessing queue health.

#Key Metrics to Monitor

#Queue Depth and Message Rates

• Queue depth — the number of messages waiting; a steady rise means consumers are overwhelmed.
• Publish rate, deliver rate, ack rate — message traffic per second.
• Unacknowledged messages — messages currently being processed; unnatural if they spike.

#Connections, Memory, and Node Health

• Connection and channel counts — a surge signals a connection leak in the application.
• Memory and disk usage — near the alarm means there's a problem.
• Consumer utilization — the percentage of time consumers are busy; determines the right prefetch.

#Prometheus Integration

#The Prometheus Plugin and Metrics Endpoint

Enable the metrics plugin and open its endpoint:

rabbitmq-plugins enable rabbitmq_prometheus
curl -s http://localhost:15692/metrics | head -n 20

The http://localhost:15692/metrics endpoint serves Prometheus-format metrics. Add it as a target in prometheus.yml:

scrape_configs:
  - job_name: rabbitmq
    static_configs:
      - targets: ["rabbitmq-node1:15692"]

#Grafana Dashboards and Alerts

Grafana provides ready-made RabbitMQ dashboards — import the dashboard with the official ID from grafana.com. From the same metrics, create alerts for queue depth, memory, and disk (this alerting is discussed in detail in episode 25). Combine with node exporter for OS metrics.

#Logging

#Log Levels and File Locations

RabbitMQ writes logs to /var/log/rabbitmq/ on native installs, or to stdout in Docker. Set the log level in rabbitmq.conf:

log.file.level = info
log.console.level = warning

For debugging, raise the level to debug; in production, info is usually enough for a normal trail.

#Structured Logging and Aggregation

Since RabbitMQ 3.8, logs can be emitted as structured logs (JSON format) for processing by centralized log systems like ELK or Loki:

log.file.formatter = json

With log.file.formatter = json, every log line becomes JSON that's easy for aggregators to parse. For audit logging, enable the rabbitmq_amqp1_0 plugin and track user access via the HTTP API or the debug logging level on authentication events.

#Conclusion

In episode 24 you read metrics from the Management UI, the HTTP API, and rabbitmqctl; integrated Prometheus with the metrics plugin; set up Grafana dashboards; and managed logging with levels, structured JSON, and centralized aggregation.

Key takeaways:

• The HTTP API and rabbitmqadmin give programmatic metric access.
• Queue depth, message rates, and unacknowledged are key queue metrics.
• Monitor connections, memory, and disk for node health.
• The rabbitmq_prometheus plugin serves metrics on port 15692.
• The official Grafana dashboard speeds up visualization.
• Logs can be leveled and formatted as JSON for aggregation.
• A good observability structure leads to reliable alerting.

In the next episode we will put together alerting and health checks — liveness and readiness checks, the health API endpoint, memory, disk, queue length, consumer down, and cluster partition alarms, plus alert integration into PagerDuty, Slack, email, and webhooks. This turns observability data into action!