Designing for Fairness in Task Scheduling

In this lesson, you will learn how to prevent "noisy neighbor" scenarios by ensuring your task queue allocates worker time proportionally across all user accounts.

The Fairness Problem

When you run a multi-tenant Node.js background worker system, you likely have users with varying levels of activity. A single user who floods the queue with 10,000 tasks can inadvertently starve other users, causing their jobs to wait indefinitely. If your workers simply process the next available task (First-In-First-Out), the system is "fair" to tasks, but unfair to users.

To achieve multi-tenant fairness, you must shift your scheduling logic from a flat queue to a Weighted Round-Robin or Account-Based Sharding strategy.

Implementing Proportional Scheduling

Instead of one global queue, you can maintain a "Ready" set of tasks organized by accountId. When a worker becomes available, it shouldn't just grab the "top" task; it should select a task from an account that has been waiting or hasn't received a high volume of recent execution time.

Consider this conceptual approach using a simple round-robin selector:

// Simple fairness selector
async function getNextTask(queuesByAccount) {
  const accountIds = Object.keys(queuesByAccount);
  
  // Sort accounts by last processing time to ensure rotation
  const sortedAccounts = accountIds.sort((a, b) => 
    queuesByAccount[a].lastProcessed - queuesByAccount[b].lastProcessed
  );

  for (const accountId of sortedAccounts) {
    if (queuesByAccount[accountId].length > 0) {
      const task = queuesByAccount[accountId].shift();
      queuesByAccount[accountId].lastProcessed = Date.now();
      return task;
    }
  }
}

Why this works

By rotating through accountIds rather than strictly following the arrival time of individual messages, you enforce a "budget" of sorts. Even if User A submits thousands of tasks, they only receive one "slot" per rotation cycle. This effectively caps the maximum throughput of any single user during periods of high contention, keeping the system responsive for everyone else.

While this adds complexity to your task retrieval logic, it transforms your worker cluster into a stable, equitable environment where system performance is no longer hostage to the behavior of a single client.

Key Takeaway: To ensure fairness, decouple your task selection from arrival time and implement a rotation strategy that grants processing slots based on account identity rather than raw queue position.