Message Loss and Ordering
What you'll learn: You'll understand why messages between distributed nodes can be lost, duplicated, or arrive out of sequence—and why this matters for system design.
The Problem with Messages in Transit
In a distributed system, nodes communicate by sending messages over a network. Unlike function calls within a single process, network messages face three fundamental challenges:
- Loss: A message might never arrive due to network failures, buffer overflows, or crashed intermediate routers
- Duplication: Retransmission logic can cause the same message to arrive multiple times
- Reordering: Messages sent in sequence A→B→C might arrive as C→A→B due to different network paths, queueing delays, or packet fragmentation
Why This Matters
Imagine you're building a banking system. If you send "withdraw 200 instead of $100. If a "confirm payment" message is lost, money disappears.
Protocol Design Requirements
Because the network is unreliable, distributed protocols must explicitly handle:
- Acknowledgments (ACKs) to detect loss
- Sequence numbers to detect duplicates and reordering
- Timeouts to trigger retransmission when ACKs don't arrive
- Idempotency so duplicate messages don't cause incorrect state changes
You cannot assume messages arrive exactly once in order—that guarantee must be built by your protocol layer.
Key Takeaway: Networks provide no guarantees about message delivery, duplication, or ordering; robust distributed systems must use sequence numbers, acknowledgments, timeouts, and idempotent operations to handle these issues explicitly.