Health Check Endpoints for Workers

In this lesson, you’ll learn how to expose internal worker state to an orchestrator like Kubernetes so it can make informed decisions about your background job infrastructure.

Why Workers Need Health Checks

When running background workers, your orchestrator (like Kubernetes) usually only knows if the process is "running." However, a process might be running while its queue connection is severed or while it’s stuck in a deadlock. If the orchestrator doesn't know the worker is unhealthy, it won't restart it, leaving your jobs sitting idle in the queue.

To bridge this gap, we expose two specific endpoints:

1. Liveness: Tells the orchestrator if the Node.js process is alive and responsive.
2. Readiness: Tells the orchestrator if the worker is actually connected to the task queue and capable of processing jobs.

Implementing the Endpoint

You can use a simple HTTP server (like http or express) running alongside your worker loop. Here is a basic implementation:

const http = require('http');

let isConnectedToQueue = false;

// Mock function checking connection to your queue (e.g., Redis)
function checkQueueHealth() {
  return queueClient.status === 'ready';
}

const server = http.createServer((req, res) => {
  if (req.url === '/healthz') {
    // Liveness check: Is the event loop blocked?
    res.writeHead(200);
    res.end('OK');
  } else if (req.url === '/ready') {
    // Readiness check: Can we actually process jobs?
    if (checkQueueHealth()) {
      res.writeHead(200);
      res.end('READY');
    } else {
      res.writeHead(503);
      res.end('NOT READY');
    }
  }
});

server.listen(8080);

Integrating with Orchestrators

By pointing your Kubernetes "Liveness Probe" to /healthz and your "Readiness Probe" to /ready, you gain automated recovery. If the readiness check fails, the orchestrator stops sending traffic or signals to the pod, allowing your monitoring system to trigger alerts while the worker attempts to reconnect. If the liveness check fails, the orchestrator will kill and restart the process, forcing a fresh connection attempt.

This visibility ensures that your background infrastructure is self-healing, preventing "silent failures" where workers appear operational but are effectively disconnected from the work.

Key Takeaway: Exposing internal status indicators like connection state to an orchestrator allows for automated, reliable recovery of your background workers when they lose contact with the job queue.