Course contentsShow
← System Design
Lesson 651 of 1,91918. Message Queues and BrokersFree lesson

Message Durability

Persisting messages to disk to survive broker restarts and ensure messages aren't lost during failures.

Message Durability

What you'll learn: How message queues persist messages to disk so they survive broker crashes and aren't lost during failures.

The Core Idea

When a producer sends a message to a message queue, you want that message to survive even if the broker (the server running the queue) crashes or restarts. Message durability means writing messages to disk storage before acknowledging receipt, ensuring they persist beyond memory and process failures.

Without durability, messages sitting in RAM would vanish if the broker crashes—your decoupled producers and consumers would lose data mid-flight.

How It Works

When a message arrives, a durable queue:

  1. Writes the message to disk (often with a write-ahead log or similar mechanism)
  2. Waits for the disk write to complete (fsync or equivalent)
  3. Then acknowledges to the producer that the message is safely stored

Only after this acknowledgment can the producer assume the message won't be lost. The message remains on disk until a consumer successfully processes it and the queue can safely delete it.

The Trade-off

Durability provides safety but costs latency and throughput. Disk writes are slower than memory operations. Non-durable (in-memory-only) queues are faster but risky—great for fire-and-forget scenarios where occasional loss is acceptable, dangerous when every message matters.

Analogy

Think of a post office: they don't just hold your package in someone's hands (memory); they put it in a secure storage room (disk) and give you a tracking receipt. Even if the postal worker goes home, your package is safe.

Key Takeaway:
Message durability persists messages to disk before acknowledgment, trading speed for the guarantee that messages survive broker failures and restarts.