Packet Fragmentation and Reassembly

In this lesson, you will learn how large data chunks are broken into smaller network packets and how Node.js streams automatically stitch that data back together for your application.

The Reality of Network Transit

When you send a large file over the internet, it doesn't travel as one giant block. Network hardware has a limit known as the Maximum Transmission Unit (MTU). If your data exceeds this limit, the network layer performs fragmentation, splitting your data into smaller packets that can travel independently across different paths to reach their destination.

Because these packets arrive at different times—and occasionally out of order—the receiving end must perform reassembly. At the OS level, the TCP stack keeps these packets in a buffer until all fragments are accounted for and sorted.

How Node.js Handles Reassembly

In Node.js, we don't manually track individual TCP packets; that is handled by the operating system’s network stack. However, since network I/O is asynchronous and unpredictable, your code receives data in "chunks" via Readable streams.

When you process a stream in Node.js, you are essentially consuming these reassembled pieces. If you are reading from a net.Socket, the data event triggers whenever a portion of the reassembled data is ready. If a message is larger than a single buffer, Node.js streams ensure that subsequent chunks are appended to the stream flow.

Example: Stream Concatenation

When dealing with fragmented application-level messages (like a JSON object spread across multiple chunks), you must concatenate them manually:

let buffer = '';

socket.on('data', (chunk) => {
  // Each chunk represents a piece of the reassembled stream
  buffer += chunk.toString();
  
  // Logic to check if the full message has arrived
  if (buffer.includes('\n')) {
    processData(buffer);
    buffer = ''; // Reset for the next message
  }
});

Because Node.js streams emit chunks as they arrive, your application logic acts as the final layer of reassembly, ensuring that you maintain the state of incoming data until you have the complete picture.

Key Takeaway: While the network stack handles low-level packet reassembly, your Node.js streams provide a reliable, sequential flow of data chunks that you must manage and concatenate to reconstruct your application-level messages.