Traffic Shedding Logic

Traffic Shedding Logic

In this lesson, you will learn how to protect your Node.js application from crashing under extreme load by intentionally rejecting low-priority requests.

When your server experiences a massive spike in traffic, system resources like CPU and memory hit their limits. If you try to process every single request, your event loop will block, latency will skyrocket, and your application may crash entirely. Traffic Shedding is the strategy of identifying "non-critical" work and dropping those requests early—before they even enter your main business logic—to preserve the health of the system for essential operations.

Implementing a Shedding Middleware

The most effective place to implement this is in an early middleware layer. By checking the current system health (such as CPU usage or the current number of active event loop tasks), you can decide whether to serve a 503 Service Unavailable status code to incoming requests.

const os = require('os');

function trafficSheddingMiddleware(req, res, next) {
  const cpuLoad = os.loadavg()[0]; // 1-minute load average
  const THRESHOLD = 5.0; // Define your "danger zone"

  // If the server is overloaded and the request is non-critical (e.g., analytics)
  if (cpuLoad > THRESHOLD && req.path === '/analytics/log') {
    return res.status(503).send('Server overloaded: Shedding non-critical traffic.');
  }

  next();
}

By filtering out background tasks like analytics logging or image processing during a high-load event, you keep your critical paths (like checkout or authentication) responsive.

Choosing What to Shed

Not all requests are created equal. When designing your shedding logic, categorize your routes:

1. Critical: Requests that drive revenue or user authentication. These are never shed.
2. Important: Requests that provide core functionality but could tolerate slight delays.
3. Non-Critical: Background telemetry, non-essential data syncs, or resource-heavy analytics. These are your first candidates for shedding.

By applying this logic, you shift from trying to do "everything for everyone" to ensuring that your server remains performant for the most important users during a crisis.

Key Takeaway: Traffic shedding is the act of proactively rejecting non-essential requests when system load exceeds safe thresholds to ensure that critical application services remain available.