Adding Jitter to Prevent Thundering Herd
What you'll learn: How randomizing retry delays prevents synchronized storms of requests that can overwhelm recovering services.
The Thundering Herd Problem
When a service fails and clients retry with exponential backoff, they often calculate identical retry delays. If 10,000 clients all fail at the same moment and use the same backoff formula (e.g., 1s → 2s → 4s → 8s), they'll all retry simultaneously at each interval. This creates synchronized waves of traffic that can repeatedly knock down a service trying to recover—like a stadium crowd stomping in unison, shaking the structure.
What Is Jitter?
Jitter is controlled randomization added to retry delays. Instead of all clients waiting exactly 4 seconds, they might wait anywhere from 3 to 5 seconds. This spreads retries across time, preventing synchronized stampedes.
Common Jitter Strategies
Full Jitter:
actual_delay = random(0, exponential_backoff)
Maximum randomization; spreads retries widely.
Equal Jitter:
actual_delay = (exponential_backoff / 2) + random(0, exponential_backoff / 2)
Balances predictability with spread.
Decorrelated Jitter:
actual_delay = random(base_delay, previous_delay * 3)
Each retry calculates based on the last attempt, creating natural variation.
Why It Works
Think of a traffic light turning green: without jitter, every car accelerates simultaneously. With jitter, they start moving at slightly different moments, preventing gridlock. Similarly, jittered retries let a recovering service handle gradually increasing load instead of instant surges.
Practical Considerations
- Add jitter after calculating exponential backoff
- Ensure jitter is truly random (not seeded identically)
- More aggressive jitter for high-traffic scenarios
- Less jitter acceptable for low-concurrency systems
Key Takeaway: Jitter breaks synchronization in retry patterns by adding randomness to delays, transforming coordinated retry storms into manageable, staggered traffic that gives failing services room to recover.