Handling Interrupted Connections

In this lesson, you will learn how to design a robust state machine that preserves your data stream's progress when a network connection drops unexpectedly.

When you stream data over a network, a disconnection doesn’t just stop the flow—it destroys the "session context," like your current byte offset or the state of a transformation. To handle this, we don't just restart; we synchronize.

The "Checkpoint and Resume" Pattern

The core idea is to treat your stream as a stateful sequence. Before you begin processing, you store a "checkpoint" (the last processed sequence ID or byte offset) in a persistent store. If the connection breaks, your application detects the error, clears the invalid buffer, and negotiates a "seek" command with the server to start precisely where the last successful chunk ended.

Here is a conceptual implementation of how to bridge that gap:

let lastProcessedId = 0;

function createResilientStream(sourceUrl) {
  const stream = connectToServer(sourceUrl, { startAt: lastProcessedId });

  stream.on('data', (chunk) => {
    // Update context as we process
    lastProcessedId = chunk.id;
    saveCheckpoint(lastProcessedId); 
    process(chunk);
  });

  stream.on('error', (err) => {
    console.error('Connection lost, attempting to resume...');
    // The stream is dead, but our session context (lastProcessedId) 
    // survives to request the data we missed.
    setTimeout(() => createResilientStream(sourceUrl), 2000);
  });
}

Why this matters

Without this logic, an interruption causes your stream to either restart from the beginning—wasting bandwidth and re-processing duplicate data—or stop permanently. By separating the Stream Transport (which is ephemeral) from the Session Context (which is persistent), you create a "self-healing" pipeline.

Remember, when you resume, you may receive duplicate data that was "in-flight" when the connection died. Your logic must be idempotent, meaning that processing the same ID twice shouldn’t corrupt your system (e.g., check if (chunk.id > lastProcessedId) before updating your database).

Key Takeaway: Never assume a stream is continuous; maintain a persistent checkpoint of your session state so you can resume exactly where you left off after an interruption.