Time-Stamping and Sequencing Packets

In this lesson, you will learn how to maintain data integrity in real-time streams by assigning sequential IDs and timestamps to your packets to detect and reorder out-of-sequence deliveries.

When you send messages over a network, particularly over unreliable connections or through load-balanced proxies, packets don't always arrive in the order they were sent. To handle this, we must shift from treating incoming data as a simple "stream" to treating it as a collection of individually identifiable packets.

The Mechanism: Sequencing and Timing

The strategy is simple: every packet sent from the server should include a metadata header containing a monotonically increasing sequenceNumber and a high-resolution timestamp.

// Server-side: Sending a sequenced packet
const sequenceManager = { counter: 0 };

function sendData(ws, payload) {
  const packet = {
    id: sequenceManager.counter++,
    timestamp: Date.now(),
    data: payload
  };
  ws.send(JSON.stringify(packet));
}

Handling Out-of-Order Packets

On the client (or receiving server), you maintain a "last processed ID" and a small buffer for out-of-order packets. If you expect ID 5 but receive ID 6, you hold 6 in a buffer and wait. If 5 arrives shortly after, you process it, then immediately drain the buffer for 6. If 5 never arrives (or is delayed beyond a threshold), you can trigger a re-request or log a data gap.

let lastProcessedId = -1;
const buffer = new Map();

function onMessage(packet) {
  if (packet.id === lastProcessedId + 1) {
    process(packet);
    lastProcessedId = packet.id;
    // Check buffer for the next sequential items
    while (buffer.has(lastProcessedId + 1)) {
      lastProcessedId++;
      process(buffer.get(lastProcessedId));
      buffer.delete(lastProcessedId);
    }
  } else if (packet.id > lastProcessedId) {
    // Packet arrived early, store for later
    buffer.set(packet.id, packet);
  }
}

By enforcing this structure, your application becomes resilient to jitter and network instability, ensuring that the state of your application remains consistent regardless of the underlying transport order.

Key Takeaway: By attaching a monotonically increasing sequence ID to every packet, you gain the ability to buffer and reorder out-of-sequence data, ensuring your application logic processes information in the exact order it was generated.