Learn RabbitMQ - CI/CD, Infrastructure as Code & GitOps

#Introduction

So far, almost all of our RabbitMQ configuration has been manual: typing rabbitmqadmin declare, rabbitmqctl set_policy, and add_user one by one. That's fast for learning, but a disaster for production. Who guarantees the staging cluster is identical to production? Who knows the last change before an incident?

The answer: Infrastructure as Code (IaC). RabbitMQ configuration is written as files that can be versioned, reviewed, and re-executed. Queues, exchanges, policies, users, and permissions are defined as JSON or code, and managed through CI/CD pipelines.

This episode takes you from JSON definitions, automation with Terraform and Ansible, message flow testing in pipelines, to GitOps — where Git becomes the single source of truth and synchronization runs automatically.

#Infrastructure as Code

#Definitions as JSON

All RabbitMQ definitions can be exported and imported as JSON. This file is a complete representation of the topology — queues, exchanges, bindings, policies, and permissions:

{
  "vhosts": [{"name": "prod"}],
  "queues": [
    {"name": "orders.q", "vhost": "prod", "durable": true,
     "arguments": {"x-queue-type": "quorum"}}
  ],
  "exchanges": [
    {"name": "orders", "vhost": "prod", "type": "topic"}
  ],
  "bindings": [
    {"source": "orders", "vhost": "prod", "destination": "orders.q",
     "routing_key": "order.*"}
  ],
  "policies": [
    {"vhost": "prod", "name": "ha", "pattern": "^orders\\.",
     "definition": {"max-length": 10000}}
  ]
}

The file above declares a vhost, a quorum queue, a topic exchange, a binding, and a policy — all in one document that can be committed to Git.

#Terraform and Ansible

Terraform has a RabbitMQ provider that declares resources idempotently. Ansible can execute rabbitmqctl commands through modules. Choose according to team culture: Terraform for cloud-resource style, Ansible for managing existing servers. The key point is that both automate the application of the JSON definitions above to every environment.

An example queue declaration with Terraform:

resource "rabbitmq_queue" "orders" {
  name    = "orders.q"
  vhost   = rabbitmq_vhost.prod.name
  settings {
    durable = true
    arguments = { "x-queue-type" = "quorum" }
  }
}

With this pattern, staging and production environments are defined from the same code — the only difference is variable values.

#CI/CD Integration

#Testing Message Flows

A CI/CD pipeline must verify that message flows work. Apply a smoke test: run a test broker, publish a message, and verify the consumer receives it. This catches regressions before they reach production:

python producer.py --queue smoke && python consumer.py --once --queue smoke

#Load Testing with PerfTest

PerfTest is the official tool for benchmarking RabbitMQ. Run it in the pipeline to compare performance before and after configuration changes:

rabbitmq-perf-test --uri amqp://localhost --queue bench \
  --producers 4 --consumers 4 --time 60

The rabbitmq-perf-test command runs publishers and consumers for 60 seconds, producing throughput and latency metrics for a baseline.

#Blue-Green and Canary Releases

For risky topology changes, use blue-green: run the new topology (green) in parallel with the old one (blue), move traffic gradually, then shut down the old one. A canary release routes a small portion of traffic to the new configuration first — if metrics look healthy, expand to the rest.

#GitOps Workflows

#Git as the Source of Truth

In GitOps, Git is the single source of truth. Every configuration change goes through a reviewed pull request; the cluster is forbidden from manual changes. Tools like ArgoCD and Flux synchronize Git state to the cluster automatically.

#Detecting Configuration Drift

If someone changes the cluster manually, the next GitOps sync will detect the drift and return the state to match Git (or alert). This is how you maintain consistency across environments without trusting human memory.

#Conclusion

In episode 30 you defined RabbitMQ topology as JSON, automated with Terraform and Ansible, tested message flows with smoke tests and PerfTest, and applied GitOps with ArgoCD, Flux, and drift detection.

Key takeaways:

• A complete topology can be declared as a single JSON file.
• Terraform and Ansible automate applying definitions to environments.
• Publish-consume smoke tests catch regressions in the pipeline.
• PerfTest provides a comparable performance baseline.
• Blue-green and canary reduce the risk of topology changes.
• Git becomes the source of truth; changes go through review.
• ArgoCD and Flux sync automatically and detect drift.

In the next episode we will put together backup, restore, and disaster recovery — exporting definitions, devising message backup strategies, recovering with definition imports, planning RPO and RTO, using Federation and Shovel for DR, and migrating between clusters without downtime. This is the last safety net before production!