In this lesson, you will learn how to maintain data integrity in distributed systems by applying Last-Write-Wins (LWW) and Conflict-free Replicated Data Types (CRDTs) to resolve concurrent write conflicts.
The Challenge of Distributed Writes
When your architecture spans multiple nodes, two users might update the same piece of data simultaneously. Without a central "source of truth," you need a deterministic strategy to reconcile these differences.
1. Last-Write-Wins (LWW)
LWW is the simplest strategy. Every write operation is assigned a high-resolution timestamp. When a conflict occurs, the system simply keeps the value associated with the latest timestamp and discards the others.
While easy to implement, LWW is "lossy"—the earlier updates are permanently deleted. It is best used for non-critical data where the most recent state is sufficient:
-- Conceptual LWW logic at the application layer
IF incoming_timestamp > stored_timestamp THEN
UPDATE vector_metadata SET embedding = new_val, updated_at = incoming_timestamp;
END IF;
2. Conflict-free Replicated Data Types (CRDTs)
Unlike LWW, CRDTs are data structures that allow updates to happen independently and concurrently on different nodes, which can then be merged mathematically without conflicts. They are designed to be "convergent," meaning all replicas will eventually arrive at the same state without needing a global lock.
Common examples include G-Counters (Grow-only counters) or LWW-Element-Sets. If you are tracking the frequency of an entity mention in a distributed vector search cluster, a CRDT ensures that adding a count on Node A and Node B eventually reflects the accurate sum across both nodes, regardless of the order of delivery.
Choosing Your Strategy
- Use LWW when your application can tolerate the loss of intermediate updates and simplicity is prioritized (e.g., updating a user's latest "preferred" embedding model).
- Use CRDTs when data integrity is paramount and you cannot afford to lose user contributions, such as collaborative document editing or distributed counters in your telemetry pipelines.
Key Takeaway: Last-Write-Wins provides a simple, lossy approach for real-time state, while CRDTs provide a sophisticated, lossless mechanism to merge distributed data into a consistent state.