Network Partition Resilience

In this lesson, you will learn how to build a "Circuit Breaker" to protect your Node.js services from cascading failures during a network partition.

Why Circuit Breakers?

When a dependency (like a database or microservice) experiences a network partition, it becomes unreachable. If your server keeps sending requests to that dead service, you end up with hundreds of pending connections, wasting memory and blocking your event loop. A Circuit Breaker acts like an electrical fuse: if the error rate exceeds a threshold, the "circuit trips" and immediately rejects subsequent requests, giving the remote service time to recover.

Implementing a Basic State Machine

You can model a circuit breaker using three states: Closed (everything is fine), Open (the circuit is broken, requests fail fast), and Half-Open (testing if the service has recovered).

Here is a simple way to track state:

let state = 'CLOSED';
let failureCount = 0;
const THRESHOLD = 5;

async function requestWithBreaker(apiCall) {
  if (state === 'OPEN') {
    throw new Error('Circuit is OPEN: Request blocked.');
  }

  try {
    const result = await apiCall();
    failureCount = 0; // Reset on success
    return result;
  } catch (err) {
    failureCount++;
    if (failureCount >= THRESHOLD) {
      state = 'OPEN';
      // Use setTimeout to transition to HALF-OPEN after a cool-down
      setTimeout(() => state = 'HALF-OPEN', 10000);
    }
    throw err;
  }
}

Preventing Cascading Failure

By "failing fast" when the circuit is OPEN, you prevent your server from stacking up requests that are destined to fail anyway. This preserves your internal resources (like memory and file descriptors) during the network partition. When the timer expires, the breaker moves to HALF-OPEN. In this state, it allows a single "probe" request to pass through. If that request succeeds, the circuit returns to CLOSED. If it fails, it trips back to OPEN.

This pattern ensures that your Node.js application remains responsive even when parts of the network infrastructure are disconnected.

Key Takeaway: Circuit breakers protect your system by failing fast during network partitions, preventing resource exhaustion and allowing dependent services time to recover.