Multiplexing Multiple Streams

Multiplexing Multiple Streams

In this lesson, you will learn how to interleave data from multiple independent sources into a single Readable stream so they can be transmitted over a single network socket.

The Core Concept

Imagine you have three different files you need to send to a server. Opening three separate TCP sockets for three small files is inefficient and consumes unnecessary resources. Multiplexing is the process of combining these multiple streams into one.

To do this, we cannot simply pipe them all to the same destination, as the data would arrive as a jumbled, unusable mess. Instead, we must use a "framing" protocol. We wrap chunks from each source in a small header that identifies which "channel" or stream the data belongs to.

Illustrating the Logic

Think of a multiplexer as a traffic controller. It reads from multiple Readable streams and writes to a single PassThrough stream. Each chunk is modified to include an ID so the receiver knows how to reassemble the pieces.

const { PassThrough } = require('stream');

function multiplex(sources) {
  const output = new PassThrough();

  sources.forEach((source, id) => {
    source.on('data', (chunk) => {
      // Wrap the chunk with a simple header: [ID_BYTE, ...CHUNK_DATA]
      const framedChunk = Buffer.concat([Buffer.from([id]), chunk]);
      output.write(framedChunk);
    });
  });

  return output;
}

Challenges to Consider

When multiplexing, you must remain mindful of the backpressure lessons we covered earlier. If you are reading from three sources at high speed but the output socket is slow, you must pause all source streams simultaneously to prevent memory overflow. Since you are now managing a single output stream, calling output.write() will return a boolean indicating if you should wait for the drain event—if it returns false, you must stop reading from all underlying sources immediately.

By interleaving data with identifiers, you transform multiple concurrent tasks into a single, efficient stream of traffic.

Key Takeaway: Multiplexing allows you to consolidate multiple data sources into a single stream by adding framing metadata, enabling efficient transmission without the overhead of maintaining multiple socket connections.