Building an Event Replay Service

In this lesson, you will learn how to design an event replay service that allows clients to recover their state seamlessly after a WebSocket disconnection.

The Problem: State Gaps

When a client loses their connection, they stop receiving real-time updates. If the server broadcasts events while the client is offline, those events are lost forever. To solve this, we don't just push events; we persist them in an append-only log, allowing the client to "catch up" upon reconnection.

Designing the Replay Log

Think of the replay service as a historical ledger. Every time a domain event occurs, we store it in a high-performance data store (like Redis or a time-series database) with a monotonically increasing sequenceNumber.

When a client reconnects, they shouldn't just ask for "all events." Instead, they send the sequenceNumber of the last event they successfully processed:

// On the server: handling the 'reconnect' event
socket.on('request-replay', async ({ lastProcessedSequence }) => {
  // Query our persistent log for all events after the client's last state
  const missingEvents = await eventLog.find({
    sequenceNumber: { $gt: lastProcessedSequence }
  });

  // Replay the missing events to the client
  missingEvents.forEach(event => {
    socket.emit('event', event);
  });
});

Key Considerations

1. Retention Policy: You cannot keep every event forever. Implement a TTL (Time-To-Live) or a buffer size (e.g., keep the last 5,000 events) to prevent your storage from growing indefinitely.
2. Idempotency: Because the replay service might deliver an event the client already processed (if their lastProcessedSequence was slightly stale), ensure the client-side state machine can handle duplicate event IDs gracefully.
3. Sequence Gaps: If a client has been disconnected for so long that their lastProcessedSequence has aged out of your retention window, the server should trigger a "full state sync" instead of an incremental replay.

By combining an append-only log with an explicit request for missed sequence numbers, you ensure that your WebSocket clients always maintain a consistent view of the domain, regardless of network instability.

Key Takeaway: An effective Event Replay Service bridges the gap between disconnection and reconnection by allowing clients to request a delta of missing updates based on their last known sequence number.