Load Shedding During Cache Failure

In this lesson, you will learn how to implement adaptive throttling to protect your primary database when your cache tier goes down, preventing a total system collapse.

The "Thundering Herd" Problem

When a distributed cache like Redis becomes unresponsive, all incoming requests that previously hit the cache will suddenly cascade to your primary database simultaneously. This sudden spike—often called a "thundering herd"—will likely exhaust your database connection pool and crash your application.

Load shedding is your circuit breaker. Instead of attempting every request, the application senses the cache failure and actively rejects a percentage of incoming traffic to keep the database alive for the remaining users.

Implementing Adaptive Throttling

We can track the health of our cache layer using a simple flag or a failure counter. When the cache is down, we transition into a "degraded mode."

let isCacheDown = false;
const FAILURE_THRESHOLD = 5;
let failureCount = 0;

async function fetchData(key) {
  if (isCacheDown && Math.random() < 0.8) {
    // Shed 80% of traffic to protect the DB
    throw new Error("Service Unavailable: Cache down, throttling enabled.");
  }

  try {
    const data = await redis.get(key);
    if (data) return data;
    
    // Fallback to database
    return await db.query(key);
  } catch (err) {
    failureCount++;
    if (failureCount > FAILURE_THRESHOLD) isCacheDown = true;
    throw err;
  }
}

In this example, when isCacheDown is true, we use Math.random() to drop 80% of incoming requests. This ensures that only 20% of the traffic reaches your database, which is often enough to serve critical requests without hitting CPU or connection limits.

Why This Matters

If you don't shed load, your database will experience 100% of the cache's former traffic. By proactively rejecting requests, you trade temporary availability for system stability. Once the cache recovers, you can reset the failureCount and stop the shedding, allowing the system to return to normal performance.

Key Takeaway: During a cache failure, shedding a portion of your traffic is a defensive necessity to prevent a database-level cascading failure.