Course contentsShow
← System Design
Lesson 1024 of 1,91926. Idempotency and RetriesFree lesson

Adding Jitter to Prevent Thundering Herd

Using randomization in retry delays to avoid synchronized retry storms that can overwhelm recovering services.

Adding Jitter to Prevent Thundering Herd

What you'll learn: How randomizing retry delays prevents synchronized storms of requests that can overwhelm recovering services.


The Thundering Herd Problem

When a service fails and clients retry with exponential backoff, they often calculate identical retry delays. If 10,000 clients all fail at the same moment and use the same backoff formula (e.g., 1s → 2s → 4s → 8s), they'll all retry simultaneously at each interval. This creates synchronized waves of traffic that can repeatedly knock down a service trying to recover—like a stadium crowd stomping in unison, shaking the structure.

What Is Jitter?

Jitter is controlled randomization added to retry delays. Instead of all clients waiting exactly 4 seconds, they might wait anywhere from 3 to 5 seconds. This spreads retries across time, preventing synchronized stampedes.

Common Jitter Strategies

Full Jitter:

actual_delay = random(0, exponential_backoff)

Maximum randomization; spreads retries widely.

Equal Jitter:

actual_delay = (exponential_backoff / 2) + random(0, exponential_backoff / 2)

Balances predictability with spread.

Decorrelated Jitter:

actual_delay = random(base_delay, previous_delay * 3)

Each retry calculates based on the last attempt, creating natural variation.

Why It Works

Think of a traffic light turning green: without jitter, every car accelerates simultaneously. With jitter, they start moving at slightly different moments, preventing gridlock. Similarly, jittered retries let a recovering service handle gradually increasing load instead of instant surges.

Practical Considerations

  • Add jitter after calculating exponential backoff
  • Ensure jitter is truly random (not seeded identically)
  • More aggressive jitter for high-traffic scenarios
  • Less jitter acceptable for low-concurrency systems

Key Takeaway: Jitter breaks synchronization in retry patterns by adding randomness to delays, transforming coordinated retry storms into manageable, staggered traffic that gives failing services room to recover.