Course contentsShow
← System Design
Lesson 438 of 1,91912. NoSQL: Wide-Column StoresFree lesson

HBase Compactions

Minor compactions merge small HFiles; major compactions merge all files and remove deleted or expired data.

HBase Compactions

What you'll learn: How HBase merges stored files to maintain performance and clean up stale data.

As data accumulates in HBase through writes, the system creates many small HFiles on disk. Over time, this fragments your data and slows down reads. Compaction is HBase's housekeeping process that merges these files together—think of it like consolidating scattered notes into organized notebooks.

Minor Compactions

Minor compactions automatically select a few small HFiles within a region and merge them into one larger file. This process runs frequently in the background to keep the number of files manageable. Since reads must check multiple HFiles (using Bloom filters), fewer files mean faster lookups.

Analogy: Imagine your desk covered with small sticky notes. A minor compaction is like gathering 5-6 notes and taping them onto a single sheet of paper—easier to scan through.

Minor compactions are:

  • Fast and lightweight
  • Don't touch all files
  • Run frequently

Major Compactions

Major compactions are more thorough—they merge all HFiles in a region into one consolidated file. Crucially, this is when HBase permanently removes deleted data (tombstones) and expired cells based on Time-to-Live (TTL) settings. Without major compactions, deleted data would persist on disk indefinitely.

Analogy: A major compaction is like spring cleaning your entire office—throwing away shredded documents (deleted data) and consolidating everything into one filing cabinet.

Major compactions are:

  • Resource-intensive (I/O and CPU)
  • Remove deleted/expired data completely
  • Run less frequently (often scheduled during off-peak hours)

Key Takeaway: Minor compactions quickly merge small HFiles for better read performance, while major compactions merge all files and permanently reclaim space by removing deleted or expired data.