Cache Stampede and Thundering Herd
What you'll learn: How to prevent multiple simultaneous cache misses from overwhelming your backend system when popular cached data expires.
The Problem
Imagine a popular cached item — say, a homepage feed with millions of viewers — expires at exactly 3:00 PM. At 3:00:01 PM, thousands of requests arrive simultaneously, all find the cache empty (cache miss), and all attempt to regenerate the data at once. Your database gets slammed with identical expensive queries, potentially causing a cascading failure.
This is the cache stampede (also called thundering herd): when a popular cache entry expires, a stampede of requests rushes to rebuild it, overwhelming the backend.
Why It Happens
- Popular data expires at a fixed time (TTL)
- High traffic hits that data immediately after expiration
- No coordination between requests — each thinks it needs to rebuild the cache
- The backend can't handle the sudden surge
Common Prevention Strategies
Probabilistic Early Expiration: Randomly expire items slightly before their actual TTL, spreading the reload across time rather than at one exact moment.
Locking/Mutex Pattern: When a cache miss occurs, the first request acquires a lock to rebuild the cache. Other requests wait for that result rather than each trying to rebuild.
Stale-While-Revalidate: Serve slightly stale data while one background process refreshes it. Users get fast responses, and only one regeneration happens.
Request Coalescing: Collapse multiple identical in-flight requests into a single backend query, sharing the result among all waiting requests.
The Analogy
Think of a theater with one ticket window. If 1,000 people arrive at once when doors open, chaos ensues. Better strategies: open doors gradually (probabilistic expiration), let one person buy tickets while others form a line (locking), or keep yesterday's tickets valid while printing new ones (stale-while-revalidate).
Key Takeaway: Cache stampedes occur when many requests simultaneously try to regenerate expired cache data; prevent them with locking, probabilistic expiration, or serving stale data during refresh.