Course contentsShow
← System Design
Lesson 632 of 1,91917. Consensus AlgorithmsFree lesson

Log Compaction: Snapshotting

How Raft discards old log entries via snapshots to prevent unbounded log growth while preserving state.

Log Compaction: Snapshotting

What you'll learn: How Raft prevents logs from growing forever by taking snapshots of committed state.


The Problem: Unbounded Log Growth

In Raft, every state change is recorded as an entry in the replicated log. Over time—days, weeks, months—this log grows without limit. Storing and replaying billions of old entries becomes impractical: disk fills up, recovery takes hours, and new servers take forever to catch up.

The Solution: Snapshots

Snapshotting is Raft's log compaction mechanism. Once the state machine has applied many committed entries, the server takes a snapshot—a compact representation of the current application state at a specific point.

Think of it like checkpointing in a video game: instead of replaying every action from the beginning, you save your progress and start fresh from that save point.

How It Works

  1. Take a snapshot: Periodically, a server serializes its current state machine state (e.g., a key-value store's entire dataset) to disk
  2. Record metadata: The snapshot includes the last included index and term—the log position it represents
  3. Discard old log entries: All entries up to and including that index can now be safely deleted
  4. Install on lagging followers: If a follower is too far behind, the leader sends the snapshot via InstallSnapshot RPC instead of individual log entries

Why It's Safe

The snapshot only includes committed state—entries that have been safely replicated to a majority. Uncommitted entries remain in the log. The snapshot essentially acts as a starting point; new entries continue to build on top of it.


Key Takeaway: Snapshotting allows Raft to discard old log entries by periodically capturing committed state, preventing unbounded log growth while preserving correctness and enabling efficient catch-up for lagging servers.