Course contentsShow
← JavaScript
Lesson 2167 of 2,72345. Workers and ConcurrencyFree lesson

Worker Pools for Reusable Threads

Implement a worker pool pattern to reuse threads for multiple tasks and reduce spawning overhead.

Worker Pools for Reusable Threads

You'll learn how to implement a worker pool pattern that reuses a fixed number of threads for multiple tasks, avoiding the overhead of repeatedly spawning new workers.

The Core Problem

Creating and destroying Worker instances for every task is expensive. Each worker initialization involves startup overhead—loading scripts, allocating resources, and establishing communication channels. If you're processing hundreds or thousands of small tasks, this overhead becomes significant.

The Worker Pool Pattern

A worker pool maintains a fixed collection of pre-initialized workers that handle tasks from a queue. Instead of spawning a new worker per task, you:

  1. Create a pool of workers once (e.g., based on CPU count)
  2. Queue incoming tasks
  3. Assign tasks to available workers
  4. Return workers to the pool when tasks complete
  5. Reuse the same workers for subsequent tasks

Think of it like a taxi stand: instead of manufacturing a new taxi for each passenger, you keep several taxis running and assign passengers to whichever taxi becomes free.

Basic Implementation Pattern

class WorkerPool {
  constructor(workerScript, poolSize) {
    this.workers = [];
    this.queue = [];
    
    for (let i = 0; i < poolSize; i++) {
      const worker = new Worker(workerScript);
      this.workers.push({ worker, busy: false });
    }
  }

  runTask(data) {
    return new Promise((resolve, reject) => {
      this.queue.push({ data, resolve, reject });
      this.processQueue();
    });
  }

  processQueue() {
    if (this.queue.length === 0) return;
    
    const available = this.workers.find(w => !w.busy);
    if (!available) return;

    const { data, resolve, reject } = this.queue.shift();
    available.busy = true;

    available.worker.postMessage(data);
    available.worker.once('message', result => {
      available.busy = false;
      resolve(result);
      this.processQueue();
    });
  }
}

Key Takeaway: Worker pools dramatically reduce thread creation overhead by maintaining reusable workers and distributing tasks from a queue, making concurrent processing both faster and more resource-efficient.