Learn RabbitMQ - Message TTL & Queue Expiration

#Introduction

Not every message deserves to live forever. A password reset notification that isn't delivered within 10 minutes is no longer useful; an expired event piling up in a queue only slows the system down. This is where Time-To-Live (TTL) comes in — a lifetime limit that makes messages "die" on their own.

TTL can be applied at two levels: per message through the expiration property, or per queue through the x-message-ttl argument. Each has different semantics and priority, and understanding this is essential for designing correct expiration behavior.

This episode also covers queue expiration for temporary queues, then the most interesting part: building delayed messaging — sending messages that may only be processed minutes or hours later — using both the TTL+DLX combination and the rabbitmq_delayed_message_exchange plugin.

#Message Time-To-Live

#Per-Message and Per-Queue TTL

Per-message TTL is set through the expiration property (in milliseconds):

Message with a 30-second TTL

properties = pika.BasicProperties(expiration="30000")
channel.basic_publish(exchange="notif", routing_key="email",
                      body=b"reset password", properties=properties)

Meanwhile, per-queue TTL is set once at declaration time:

rabbitmqadmin declare queue name=otp_queue \
  arguments='{"x-message-ttl":60000}'

The rabbitmqadmin declare queue command above makes every message in otp_queue expire after 60 seconds.

#TTL Priority: Message vs Queue

If a message has its own expiration and its queue also has an x-message-ttl, then the smallest value applies. This rule matters: per-message TTL can be shorter (giving the message a shorter life) but never longer than the queue's TTL.

#Handling Expired Messages

An expired message is removed from the queue and, if configured, forwarded to a DLX (not requeued). Note that TTL is only counted while the message is in the queue — a message being held by a consumer is not counted. For this reason, make sure the TTL isn't shorter than the processing time the consumer needs.

#Queue Expiration

#x-expires: Auto-Delete After Inactivity

The x-expires argument deletes the queue automatically if the queue is unused for a certain duration — no consumers, no publishes, and no gets. The time is counted in milliseconds:

rabbitmqadmin declare queue name=temp_ingest \
  arguments='{"x-expires":600000}'

This feature is very useful for temporary queues like RPC callback queues that are only needed during a session. The queue is deleted along with all its messages, so make sure there's no important data.

#Temporary Queue Use Cases

The combination of auto_delete, exclusive, and x-expires fits: RPC callback queues, per-request queues in batch operations, and event topologies only needed for a certain period of time.

#Delayed Messages

#The TTL + DLX Delay Pattern

Classic RabbitMQ has no native delay feature, but it can be simulated: publish a message to a queue with x-message-ttl, and after the TTL expires the message is re-routed to the destination queue via a DLX. The result: the message is only processed after the desired delay.

rabbitmqadmin declare exchange name=jobs.dlx type=direct
rabbitmqadmin declare queue name=jobs.delay \
  arguments='{"x-message-ttl":300000,"x-dead-letter-exchange":"jobs.dlx","x-dead-letter-routing-key":"go"}'
rabbitmqadmin declare queue name=jobs.main durable=true
rabbitmqadmin declare binding source=jobs.dlx destination=jobs.main routing_key=go

A message entering jobs.delay "sleeps" for 5 minutes, then is forwarded to jobs.main.

#The Delayed Message Exchange Plugin

For more flexible delays (one exchange for many delay durations), use the rabbitmq_delayed_message_exchange plugin:

rabbitmq-plugins enable rabbitmq_delayed_message_exchange

Then declare an exchange of type x-delayed-message and send messages with a delay header:

Publish with the x-delay header

channel.exchange_declare(
    exchange="delayed", exchange_type="x-delayed-message",
    arguments={"x-delayed-type": "direct"},
)
properties = pika.BasicProperties(headers={"x-delay": 10000})
channel.basic_publish(exchange="delayed", routing_key="tasks",
                      body=b"proses nanti", properties=properties)

The x-delay header in milliseconds makes the message forwarded to the binding after 10 seconds. This plugin is supported by many client SDKs, but keep in mind the queue must stay alive throughout the delay.

#Conclusion

In episode 12 you mastered per-message and per-queue message TTL, the smallest-TTL-wins priority, expired message handling, queue expiration with x-expires, and two delayed messaging approaches: the TTL+DLX pattern and the delayed message exchange plugin.

Key takeaways:

• Per-message TTL via the expiration property; per-queue TTL via x-message-ttl.
• The smallest TTL value applies when both are set.
• Expired messages are discarded and can be forwarded to a DLX.
• TTL is only counted while the message is in the queue, not in the consumer.
• x-expires deletes queues unused for a certain duration.
• Delay without a plugin = queue TTL + DLX to the destination queue.
• The rabbitmq_delayed_message_exchange plugin provides flexible per-message delays.

In the next episode we will set up virtual hosts and multi-tenancy — separating resources within a single RabbitMQ instance, creating vhosts per environment and per tenant, managing per-vhost access, and naming, monitoring, and backup strategies for each vhost. This is the foundation of tenancy isolation in RabbitMQ!