Load Shedding in Async Flows

In this lesson, you will learn how to proactively reject new tasks when your system reaches its capacity to prevent crashes and resource exhaustion.

The Concept of Load Shedding

When your Node.js application is under heavy traffic, it might start queuing up more asynchronous tasks than it can actually finish. If you keep accepting work without limit, your memory usage spikes, response times balloon, and eventually, the event loop becomes overwhelmed.

"Load shedding" is the act of checking if your system is currently "too busy" before accepting a new task. Instead of trying to process everything and risking a system-wide collapse, you intentionally drop or reject incoming requests. It is better to fail fast on a few requests than to have the entire system fail for everyone.

Implementing a Basic Load Shedder

To implement this, you keep track of your active tasks and compare them against a defined maximum threshold. If the count exceeds the threshold, you immediately trigger an error callback to inform the requester that the system is at capacity.

let activeTasks = 0;
const MAX_CONCURRENT_TASKS = 5;

function processTask(data, callback) {
  // Check capacity before accepting the work
  if (activeTasks >= MAX_CONCURRENT_TASKS) {
    return callback(new Error("Server Busy: Load shedding in effect."));
  }

  activeTasks++;
  
  // Simulate async work
  setTimeout(() => {
    activeTasks--;
    callback(null, `Processed: ${data}`);
  }, 1000);
}

// Example usage
for (let i = 0; i < 7; i++) {
  processTask(`Job ${i}`, (err, result) => {
    if (err) console.error(err.message);
    else console.log(result);
  });
}

In this code, when the loop tries to fire the 6th and 7th jobs, the if condition detects that activeTasks has hit the limit. Instead of adding those jobs to a growing queue, it returns an error immediately. This protects your application by ensuring that the number of concurrent operations never exceeds the performance limits you have set, maintaining the stability of your event loop and memory.

Key Takeaway: Load shedding preserves system stability by actively rejecting incoming tasks once a defined concurrency threshold is reached, trading individual request failures for overall service availability.