Managing Cache Re-population Storms

In this lesson, you will learn how to implement request queueing to prevent "Cache Stampedes" from overwhelming your database when a popular key expires.

The Problem: Cache Stampedes

When a highly requested key (like a homepage configuration or a trending product) expires, your Node.js application might receive thousands of concurrent requests before the cache can be refilled. If every request sees a "cache miss" simultaneously, they will all attempt to query the database at the same time. This redundant load is known as a cache re-population storm, which can lead to high latency or even a database outage.

The Solution: Request Queueing

Instead of letting every request hit the database, we can use a "locking" or "queueing" strategy. When the first request detects a cache miss, it places a temporary lock (or a "promise" in memory) in the cache. Subsequent requests for that same key are instructed to wait for the result of the first query rather than initiating their own.

Here is how you can implement a basic locking mechanism using an async queue approach:

const inflightRequests = new Map();

async function getCachedData(key) {
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);

  // If a request is already in progress, wait for it
  if (inflightRequests.has(key)) {
    return inflightRequests.get(key);
  }

  // Otherwise, create a new request promise
  const requestPromise = (async () => {
    try {
      const data = await db.query('SELECT * FROM items WHERE id = ?', [key]);
      await redis.setex(key, 3600, JSON.stringify(data));
      return data;
    } finally {
      // Remove from map once finished so future requests can start fresh
      inflightRequests.delete(key);
    }
  })();

  inflightRequests.set(key, requestPromise);
  return requestPromise;
}

By storing the promise in the inflightRequests Map, we ensure that only the first request performs the expensive database operation. Every other request arriving while the first is still pending will simply await that same promise, effectively collapsing the storm into a single database hit. This keeps your application stable even under extreme traffic bursts.

Key Takeaway: Use in-memory promise tracking to collapse concurrent cache misses into a single database query, shielding your backend from redundant re-population traffic.