Streaming Audio and Video Processing

In this lesson, you will learn how to process media fragments efficiently by building a streaming pipeline that avoids loading entire video or audio files into your server's memory.

The Challenge of Media Size

When dealing with media like high-definition video or lossless audio, files are often gigabytes in size. If you attempt to load a file into a variable using fs.readFile, you will quickly exceed the Node.js memory limit, causing your application to crash. Instead, we treat media as a continuous flow of binary chunks, processing and piping them as they arrive.

Building the Fragment Pipeline

The secret to efficient media processing is to think of a file not as a static object, but as a series of small, manageable "fragments." We use the stream.pipeline utility to connect a Readable stream (the source file) through one or more Transform streams (your processing logic) to a Writable stream (the destination or network response).

Because we are using streams, the system only keeps the current chunk in memory. If your Transform stream is slower than the file read speed, the Readable stream will automatically pause, preventing memory overflow.

Concrete Example: Adding a Watermark/Filter

Imagine you want to pipe an incoming video stream through a processor:

const { pipeline } = require('stream/promises');
const fs = require('fs');

async function processMedia(inputPath, outputPath) {
  await pipeline(
    fs.createReadStream(inputPath), // Source: Read chunk-by-chunk
    new MediaProcessorTransform(),  // Transform: Modify data on-the-fly
    fs.createWriteStream(outputPath) // Destination: Save to disk
  );
  console.log('Pipeline finished without loading the whole file!');
}

In this setup, MediaProcessorTransform inherits from the Transform class. As data chunks enter the _transform method, you manipulate the binary buffer (e.g., modifying audio gain or adjusting frame headers). Once the buffer is processed, you push() it forward to the next stage of the pipeline. By the time the next chunk arrives, the previous one has already been flushed to the Writable stream, keeping your memory usage constant regardless of whether the file is 10MB or 10GB.

Key Takeaway: By connecting readable, transform, and writable streams into a single pipeline, you process media as a continuous flow, ensuring your server remains stable even when handling massive video and audio files.