Read Repair and Anti-Entropy
What You'll Learn
You'll learn two mechanisms that distributed systems use to detect and fix inconsistencies between replicas automatically over time.
The Problem
In leaderless replication systems (where any replica can accept writes), replicas can fall out of sync—perhaps due to network partitions, temporary node failures, or concurrent writes. Once the system is back to normal, these inconsistencies need to be resolved.
Read Repair
Read Repair fixes inconsistencies opportunistically during read operations.
When a client reads data, the system queries multiple replicas. If the responses differ (detected by comparing version numbers or timestamps), the system:
- Returns the most recent value to the client
- Writes that newest value back to any out-of-date replicas
Analogy: Imagine three librarians with copies of a catalog. When a patron asks for a book's location, all three respond. If two say "Shelf A" (updated) and one says "Shelf B" (outdated), the patron gets "Shelf A" and the outdated librarian's catalog is corrected on the spot.
Limitation: Read repair only fixes data that gets read. Rarely-accessed data might stay inconsistent indefinitely.
Anti-Entropy
Anti-Entropy is a background process that continuously compares replicas and synchronizes differences.
Replicas periodically exchange information (often using Merkle trees for efficiency) to identify mismatches, then exchange and reconcile divergent data.
Analogy: Like a scheduled inventory audit where librarians systematically compare their entire catalogs, not just the books people ask about.
Trade-off: Anti-entropy catches everything but consumes network and CPU resources continuously.
Used Together
Most systems employ both: read repair provides fast, low-overhead correction for hot data, while anti-entropy ensures eventual consistency for all data, including cold data that's rarely accessed.
Key Takeaway: Read repair fixes inconsistencies during normal reads, while anti-entropy uses background processes to systematically reconcile all replicas—together they ensure eventual consistency in leaderless replication systems.