Read Path and Compaction
What you'll learn: How BigTable retrieves data by merging in-memory and on-disk structures, and why periodic compaction keeps the system efficient.
How Reads Work in BigTable
When you read data from BigTable, the system must check multiple places because writes go to memory first, then flush to disk over time.
The read path follows this sequence:
- Check the MemTable — the in-memory structure holding recent writes
- Check SSTables on disk — immutable files stored in GFS, potentially many of them
- Merge results — newer values override older ones, with timestamps determining precedence
Think of it like searching for the latest version of a document: you check your desk (MemTable) first, then filing cabinets (SSTables) in reverse chronological order, taking the newest version you find.
Why Compaction Matters
As MemTables flush repeatedly, many SSTables accumulate. More files mean slower reads — you must search through more places to find data.
Compaction solves this by periodically merging multiple SSTables into fewer, larger files:
- Minor compaction — merges a few recent SSTables
- Major compaction — merges all SSTables for a tablet into one file, removing deleted entries and old versions
Compaction happens in the background. It reduces read amplification (number of files checked per read) and reclaims storage by eliminating tombstones and outdated versions.
The trade-off? Compaction uses I/O and CPU resources, so it must be balanced with serving traffic.
Key Takeaway: Reads in BigTable merge data from the MemTable and multiple SSTables, while compaction periodically consolidates SSTables to maintain read performance and reclaim space.