Rate Limiting and Throttling Background Work

In this lesson, you will learn how to implement window-based constraints to protect external services from being overwhelmed by your background worker's job processing speed.

When your worker processes tasks—such as sending API requests to a third-party payment gateway or a social media platform—it often runs much faster than the receiving service can handle. If you push too many jobs at once, the external service will return "429 Too Many Requests" errors, potentially leading to IP bans or temporary blacklisting. To prevent this, we use Window-based Rate Limiting.

The Fixed Window Approach

The simplest form of rate limiting is the "Fixed Window" algorithm. You define a time window (e.g., 60 seconds) and a maximum number of allowed requests (e.g., 100 requests). As your background worker picks up jobs from your queue, it keeps a counter of how many requests it has sent within the current window.

Consider this basic implementation pattern using an in-memory counter:

let requestCount = 0;
const MAX_REQUESTS = 100;
const WINDOW_MS = 60000;

// Reset the window every minute
setInterval(() => { requestCount = 0; }, WINDOW_MS);

async function processJob(job) {
  if (requestCount >= MAX_REQUESTS) {
    // Throttling: requeue the job to process later
    return requeueJob(job);
  }

  requestCount++;
  await sendExternalRequest(job.data);
}

Why Throttling Matters

In a distributed environment, you might have multiple workers running. Relying solely on requestCount inside one process won't be enough because the external service sees the sum of all your workers. In production, you would typically use a shared store like Redis to track the global requestCount across all nodes.

When your worker hits the limit, throttling is the correct behavior: you don't discard the task; you delay it. By moving the task to a "delayed" queue or simply waiting for the window to reset, you ensure your system remains a good "digital citizen," maintaining the integrity of your connection to the external API without causing service disruptions.

Key Takeaway: Use window-based constraints to batch your outbound tasks and throttle job execution to match the throughput limits of external services, preventing failures before they happen.