Preventing Cache Stampede
What you'll learn: How to protect your system when a popular cache key expires and many requests try to rebuild it simultaneously.
The Problem Recap
You learned in the previous lesson that a cache stampede happens when a cached item expires and many clients simultaneously rush to regenerate it, overwhelming your backend. Now let's explore the techniques that prevent this chaos.
Core Prevention Techniques
Request Coalescing (Collapsing)
When multiple requests arrive for the same missing cache key, only one request actually queries the backend—the rest wait for that result. Think of it like carpooling: when everyone needs to go to the same destination, only one car makes the trip while others ride along.
Probabilistic Early Expiration
Instead of waiting for TTL to fully expire, recalculate the cache entry slightly before expiration—but only probabilistically. As expiration approaches, the probability increases. This spreads out regeneration over time rather than having everything expire at once.
probability = random(0,1) < (time_since_fetch / TTL)
if probability: regenerate_now()
Locking Mechanisms
When a cache miss occurs, acquire a distributed lock before regenerating. If the lock is already held, other requests either wait briefly or serve stale data. The lock ensures only one process rebuilds the cache.
Key Takeaway: Preventing cache stampede requires coordination—either collapse duplicate requests into one (coalescing), stagger regeneration timing (probabilistic expiration), or use locks to ensure single-threaded rebuilds. These techniques transform a thundering herd into an orderly queue.