Designing Eventual Consistency Aggregators

In this lesson, you will learn how to design an "Eventual Consistency Aggregator" to merge disparate asynchronous status updates into a single, coherent view for your users.

The Problem: Fragmented State

When building distributed systems, a user’s request (like "Create Order") often triggers multiple background processes—inventory reservation, payment processing, and shipping initiation. Each of these emits its own status update at different times. If you serve the user the raw state from each individual service, they see a fragmented, confusing experience.

An Aggregator sits between your background workers and your read-only database (or cache), acting as the "single source of truth" for the user’s dashboard.

How it Works

Instead of the user polling ten different services, your aggregator consumes events from a message queue. As each message arrives, the aggregator performs an "upsert" operation on a projection model—a unified JSON document specifically shaped for the UI.

Consider this simplified Node.js worker pattern:

// A simple Aggregator worker listening to multiple queues
queue.on('message', async (msg) => {
  const { orderId, status, timestamp, type } = JSON.parse(msg.body);

  // Update the unified view for this specific order
  await db.orders.updateOne(
    { orderId },
    { 
      $set: { [`statusUpdates.${type}`]: status, lastUpdated: timestamp },
      $currentDate: { lastModified: true }
    },
    { upsert: true }
  );
});

Designing the Aggregator

To make this robust, follow these two rules:

1. Idempotent Merging: Since messages might arrive out of order (due to network jitter), your aggregator should use the timestamp field within the message to ensure that an "Old" update doesn't overwrite a "New" status. Use lastUpdated checks in your where clause: db.orders.updateOne({ orderId, lastUpdated: { $lt: incomingTimestamp } }, ...).
2. Projection Model: Don't expose your raw database schema. Create a dedicated "Read Model" that maps service statuses into human-readable flags (e.g., READY, IN_PROGRESS, FAILED). This allows you to decouple your internal event vocabulary from what the user actually needs to see.

By offloading this logic to an aggregator, you provide a consistent, performant UI experience, even if the underlying background tasks finish seconds apart.

Key Takeaway: An Eventual Consistency Aggregator consumes distributed events to construct a unified, read-optimized state, ensuring users see a coherent status while background workers process tasks independently.