Stream-based Circuit Breakers

In this lesson, you will learn how to protect your Node.js infrastructure by implementing a circuit breaker that halts stream processing when downstream services become unresponsive.

The Problem: Cascading Failures

When you pipe data through a long pipeline, a failing downstream service (like a database or an external API) can lead to a build-up of unhandled data. If the service is failing, retrying blindly just adds more pressure, potentially crashing your application or the target service. A Stream-based Circuit Breaker acts as a "safety fuse" inside your pipeline.

Implementing the Breaker

A circuit breaker exists in one of three states: Closed (everything is fine), Open (the service is failing; stop traffic), and Half-Open (test if the service has recovered).

In a Transform stream, you can intercept data chunks and verify the circuit state before calling the callback(). If the circuit is "Open," you shouldn't process the data; instead, you should treat the chunk as a failure or pause the pipeline.

const { Transform } = require('stream');

class CircuitBreaker extends Transform {
  constructor() {
    super({ objectMode: true });
    this.state = 'CLOSED'; // 'CLOSED', 'OPEN', 'HALF-OPEN'
    this.failureCount = 0;
  }

  _transform(chunk, encoding, callback) {
    if (this.state === 'OPEN') {
      // Reject the chunk immediately
      return callback(new Error('Circuit is open: downstream service unavailable'));
    }

    // Process and track success/failure
    try {
      // Simulate calling a downstream service
      this.push(chunk);
      this.failureCount = 0;
      callback();
    } catch (err) {
      this.failureCount++;
      if (this.failureCount > 5) this.state = 'OPEN';
      callback(err);
    }
  }
}

Why Streams Benefit

By placing this logic directly within the stream, you leverage backpressure. When the circuit breaker emits an error, the upstream source (the producer) is signaled to stop pushing data. This prevents memory bloat because you aren't buffering thousands of pending requests in an attempt to hit a broken server. The stream stops, your error handling logic triggers, and you preserve the health of the event loop.

Key Takeaway: A stream-based circuit breaker prevents system collapse by proactively closing the pipe when downstream failures cross a threshold, naturally utilizing backpressure to halt data flow.