In-Sync Replicas (ISR)
What you'll learn: How Kafka tracks which replicas are "caught up" to ensure data durability and consistency.
The Core Problem
In a distributed system like Kafka, the leader partition writes data, and follower replicas copy it. But network delays, slow disks, or temporary failures mean followers might lag behind. If the leader crashes, how do you know which follower is safe to promote? You need to track which replicas are actually synchronized.
What Is the ISR Set?
The In-Sync Replica (ISR) set is Kafka's list of replicas—including the leader—that are fully caught up with the latest messages. A replica stays in the ISR if it:
- Regularly fetches new data from the leader
- Hasn't fallen too far behind (configurable threshold)
If a follower lags beyond acceptable limits, Kafka removes it from the ISR until it catches up.
Why ISR Matters
The ISR set is critical for durability guarantees:
- Producer acknowledgments: When a producer requests
acks=all, Kafka waits until all replicas in the ISR have written the message before confirming success. This prevents data loss even if the leader immediately fails. - Leader election: Only replicas in the ISR are eligible to become the new leader, ensuring the new leader has all committed messages.
- Consistency: By tracking sync status, Kafka balances availability (allowing some replicas to lag temporarily) with consistency (only promoting caught-up replicas).
Analogy
Think of a classroom where the teacher (leader) writes notes on the board. Students (followers) copy them into notebooks. The ISR is the list of students whose notebooks are up-to-date. If the teacher leaves, only a student from that list can take over teaching—guaranteeing no lessons are lost.
Key Takeaway:
The ISR set identifies which replicas are fully synchronized with the leader, ensuring Kafka can provide strong durability and consistency guarantees during both normal operation and failover.