Replication Lag and Consistency
What you'll learn: How replication lag impacts data consistency and the challenges of reading data you've just written in distributed database systems.
Understanding Replication Lag
In a Primary-Replica Architecture, changes written to the primary database must propagate to replica servers. Replication lag is the time delay between when data is written to the primary and when it appears on the replicas. This lag can range from milliseconds to seconds (or longer during network issues).
Think of it like breaking news: the TV station (primary) broadcasts first, but cable providers (replicas) might be a few seconds behind. Viewers watching different providers see the news at slightly different times.
Eventual Consistency
With replication lag, systems operate under eventual consistency—replicas will eventually match the primary, but aren't guaranteed to be identical at any given moment. This creates scenarios where:
- A user writes data to the primary
- The application immediately reads from a replica
- The replica hasn't received the update yet
- The user sees stale data (or their write appears to have "disappeared")
Read-After-Write Consistency Challenges
The read-after-write problem is particularly frustrating: "I just updated my profile photo, why do I still see the old one?" This happens when writes go to the primary but subsequent reads hit a lagging replica.
Common mitigation strategies include:
- Reading user's own data from the primary
- Tracking write timestamps and avoiding stale replicas
- Session-based routing to ensure consistency within a user session
Monitoring Replication Lag
Key metrics to track:
replication_lag_seconds: time behind primaryreplica_lag_bytes: data volume not yet applied- Monitoring trends to detect replication issues before user impact
Key Takeaway: Replication lag creates eventual consistency where replicas temporarily diverge from the primary, requiring careful design to prevent users from reading stale data immediately after writes.