Learn RabbitMQ - Performance Tuning & Optimization

#Introduction

The same broker can feel very different depending on how you use it. An app that publishes one message per new connection, and another that uses a shared channel with batching, can differ tenfold in throughput. In this episode we settle the performance side.

RabbitMQ performance optimization can be split into four layers: publisher, consumer, queue, and system. Each has levers you can turn — and almost always, the highest-impact optimization is on the application side, not broker configuration.

Remember the golden rule: measure first, optimize later. Don't guess; use tools like PerfTest (episode 30) to get a baseline before and after every change.

#Publisher Optimization

#Batching and Async Publishing

A publisher that sends one message then waits for a per-message confirmation is very slow. Batching — collecting several messages before sending — dramatically reduces the number of round-trips. Combine it with async confirms: send the batch, then process confirmations in a callback.

Publish sequentially without waiting for confirm

channel.confirm_delivery()
for i in range(1000):
    channel.basic_publish(exchange="", routing_key="bulk", body=f"m{i}".encode())
# wait for all confirms to finish
connection.process_data_events()

The pattern above publishes 1000 messages in one wave, and confirms are handled asynchronously by the broker — far faster than waiting one by one.

#Reusing Connections and Channels

TCP connections are expensive; channels are cheap. Reuse connections as long as possible, and don't create a new connection per message. For high throughput, use several channels in line with your worker threads.

One often-mistaken pattern: creating a new connection inside a publish loop. Every connection needs a TCP handshake, an AMQP capability exchange, and the allocation of an Erlang process on the broker. If an app publishes 1000 messages with 1000 new connections, it can be dozens of times slower than one connection with 1000 sequential publishes. Always create the connection at application startup and share its channels.

#Consumer Optimization

#Tuning Prefetch and Parallelism

A prefetch_count that's too small makes consumers sit idle waiting for messages; too large makes one consumer hoard messages. Measure and adjust. Also add parallelism: several consumer workers per channel or several channels at once increases CPU utilization.

One effective pattern: use a worker pool on the consumer side — one channel receives messages, then distributes them to several goroutines or threads for parallel processing. A sufficiently large prefetch combined with a worker pool often raises CPU utilization without adding new connections.

#Auto-Ack vs Manual Ack

auto_ack gives the highest throughput because the broker doesn't wait for confirmations, but it risks losing messages when a consumer crashes. manual ack is safer with a small overhead. This trade-off is a business decision: how much is the processed data worth compared to the cost of loss.

If you still choose auto-ack for speed, provide compensation: route failed messages to a dead letter exchange (episode 11), or make sure consumers are idempotent so reprocessing after a crash doesn't corrupt data. That way you keep the speed without losing the error trail.

#Queue Optimization

#Choosing the Queue Type and Lazy Mode

The queue type choice strongly determines performance: classic for simple single-node setups, quorum for reliability, stream for replay throughput. For queues that tend to pile up large, consider lazy mode, which writes messages to disk earlier:

rabbitmqadmin declare queue name=big_queue \
  arguments='{"x-queue-type":"classic","x-queue-mode":"lazy"}'

x-queue-mode=lazy makes messages written directly to disk, reducing the risk of running out of memory for long queues. Full lazy queue details are in episode 23.

#Queue Length and Message Size

Set x-max-length so queues don't grow out of control, and shrink message size: avoid storing large data in the body, compress payloads (episode 9), and use object storage references for files.

#System-Level Tuning

#Erlang VM and Memory

Adjust the memory high watermark so the broker warns earlier:

vm_memory_high_watermark.relative = 0.7
disk_free_limit.relative = 1.5

vm_memory_high_watermark at 0.7 means the memory alarm starts at 70 percent usage — leaving safe headroom before a crash.

#File Descriptors, Disk, and Network

• File descriptors: raise the ulimit — every connection needs several FDs.
• Disk I/O: use SSDs for a busy queue store; slow disks throttle persistent throughput.
• Network buffer: raise the TCP buffer size (tcp_bufsize) for high-latency links.

To check whether file descriptors are the bottleneck, compare the file descriptors currently in use against the limit:

rabbitmqctl status | grep -A 4 -i "file descriptors"

If the number in use approaches the limit, raise the system ulimit by editing /etc/security/limits.conf and restarting the service. Remember: exhausted file descriptors cause new connections to be rejected — and the symptoms are often confusing because they don't look like a broker error.

#CPU Affinity and NUMA

On machines with many cores and a NUMA architecture, the Erlang scheduler allocation can be optimized by setting CPU affinity. This configuration is rarely needed for small machines, but on nodes with many cores and high load, adjusting +sbwt and +sbwtdcpu in the Erlang VM arguments can reduce cache misses and scheduler contention. Only do this after measurements show the scheduler is the bottleneck, not based on guesses.

#Benchmarking and Comparison

#Latency vs Throughput

Optimization often faces you with a latency vs throughput choice. Batching, for example, raises throughput but adds a little latency because messages are held waiting for a full batch. Conversely, one-by-one publishing minimizes latency but has low throughput. Which to prioritize depends on the business: trading systems chase latency, while data pipelines chase throughput. Write down target numbers upfront — for example "5000 publishes per second with p95 latency under 20 ms" — then measure against that target.

#One Change at a Time

Because many variables affect each other, changing several configurations at once makes results un-attributable. Make one change, measure, record, then move to the next change. Save a baseline before starting, and keep the measurement environment stable — running benchmarks on a laptop with other apps open will produce misleading numbers.

#Conclusion

In episode 22 you optimized publishers with batching and async confirms, tuned prefetch and consumer parallelism, chose queue types and lazy mode, and did system-level tuning for the Erlang VM, file descriptors, disk, and network.

Key takeaways:

• Batching and async confirms are the highest-impact publisher optimizations.
• Reuse connections; create new channels when you need concurrency.
• Prefetch is tuned based on message size and processing speed.
• Auto-ack is fast but fragile; manual ack is safe with small overhead.
• Lazy mode writes messages to disk earlier for large queues.
• Set queue length limits and shrink message size.
• Raise the file descriptor ulimit and use SSDs for disk performance.

In the next episode we will manage memory and resource management — memory and disk alarms, memory calculation strategies, paging messages to disk, lazy queues, connection, channel, and queue limits per vhost, and log management and message store compaction. This is the broker's defense against shrinking resources!