Handling Cache Stampede on Popular URLs
You'll learn how to prevent thousands of servers from simultaneously hammering your database when a popular cached URL expires.
The Problem
A cache stampede (also called "thundering herd") occurs when a highly popular short URL's cache entry expires. Suddenly, hundreds or thousands of servers realize they don't have the cached value. They all race to the database simultaneously to fetch the same URL mapping, creating a massive spike in database load that can bring your system down.
Think of it like a stadium of people all rushing through a single door the moment it opens—chaos and congestion.
Probabilistic Early Expiration
Instead of waiting for TTL to hit exactly zero, recompute the cache entry probabilistically before expiration:
# Pseudo-code concept
def should_refresh_early(ttl_remaining, beta=1.0):
random_value = random.uniform(0, 1)
delta = ttl_remaining * beta * log(random_value)
return delta < 0
When ttl_remaining is low, the probability of early refresh increases. This spreads out the refresh load over time rather than concentrating it at one moment.
Request Coalescing with Locking
Only let one request through to the database per cache miss:
- First request acquires a distributed lock (e.g., Redis
SETNX) - Other concurrent requests wait or use stale data briefly
- Once the first request updates the cache, waiting requests use the fresh value
This prevents 1,000 requests from becoming 1,000 database queries—they become 1 query and 999 cache hits.
Combining Strategies
Use both: probabilistic expiration reduces stampede likelihood, while locking handles the rare cases where multiple servers miss simultaneously.
Key Takeaway:
Prevent cache stampede by refreshing popular entries probabilistically before expiration and using distributed locks to ensure only one request per cache miss hits the database, protecting it from sudden traffic surges.