Concurrency Limits in Worker Threads

In this lesson, you will learn how to prevent system overload by placing a strict concurrency limit on the number of active Worker threads running CPU-intensive tasks.

The Problem: Thread Exhaustion

In Node.js, the main event loop is responsible for handling I/O. If you have a heavy task—like complex data processing or image manipulation—you move that work to a Worker thread to avoid "Event Loop Starvation." However, spawning a new thread for every single request is dangerous. If 1,000 requests arrive, creating 1,000 threads will overwhelm your CPU and crash your application. This is where concurrency limiting becomes essential.

Managing a Worker Pool

To keep your application stable, you must implement a "pool" pattern. Instead of letting tasks run wild, you create a fixed number of workers (e.g., matching the number of CPU cores) and maintain a queue of pending tasks.

Think of it like a bank: if there are 10 bank tellers (your worker limit) and 50 customers (your incoming tasks), the customers don't all crowd the desks at once. They wait in a line (a queue) and move forward only when a teller becomes free.

Here is a simplified look at how you might track this:

const activeWorkers = new Set();
const taskQueue = [];
const MAX_CONCURRENCY = 4;

function runTask(task) {
  if (activeWorkers.size >= MAX_CONCURRENCY) {
    // Add to queue if we are at capacity
    taskQueue.push(task);
    return;
  }

  const worker = spawnWorker(task); // Imagine this creates a new Worker
  activeWorkers.add(worker);

  worker.on('exit', () => {
    activeWorkers.delete(worker);
    // When one finishes, process the next in line
    if (taskQueue.length > 0) {
      runTask(taskQueue.shift());
    }
  });
}

By checking the size of your activeWorkers set before starting a new thread, you effectively throttle the load. This ensures your CPU stays at a manageable level, keeping the event loop responsive even under heavy demand. By combining this with the event-based patterns we’ve already studied, you can build systems that remain performant regardless of incoming volume.

Key Takeaway: Never spawn an unlimited number of workers; use a queue and a counter to ensure that your active thread count never exceeds your available CPU resources.