Course contentsShow
← System Design
Lesson 436 of 1,91912. NoSQL: Wide-Column StoresFree lesson

HBase Write Path and WAL

Writes go to the Write-Ahead Log (WAL) for durability, then to MemStore in memory, eventually flushed to HFiles.

HBase Write Path and WAL

What You'll Learn

You'll understand how HBase ensures data durability during writes using a Write-Ahead Log before storing data in memory.

The Write Path Journey

When a write request arrives at HBase, data doesn't go directly to disk—it takes a carefully orchestrated path designed for both speed and safety.

Think of it like a restaurant kitchen: When a customer orders food, the waiter first writes the order on a ticket (backup) before shouting it to the chef (active processing). If the chef forgets, the written ticket ensures nothing is lost.

The Three-Step Write Process

  1. Write-Ahead Log (WAL): First, the write is appended to the WAL, a sequential log file stored on disk. This ensures durability—even if the server crashes immediately after, the write is preserved. The WAL acts as HBase's safety net.

  2. MemStore: After the WAL confirms the write, data is written to the MemStore, an in-memory buffer for each column family. This provides fast read access to recent writes without hitting disk.

  3. Flush to HFiles: When the MemStore fills up (typically 128MB-256MB), it's flushed to disk as an immutable HFile. This converts the temporary in-memory data into permanent storage.

Why This Matters

The WAL is critical for crash recovery. If a Region Server fails, HBase can replay the WAL to reconstruct any MemStore data that hadn't been flushed yet. Without it, all in-memory writes would be lost.

The MemStore enables write batching—many small writes accumulate in memory and flush together efficiently, avoiding the overhead of many tiny disk writes.

Key Takeaway: HBase writes data to the Write-Ahead Log first for durability, then to the in-memory MemStore for speed, and finally flushes to permanent HFiles on disk—balancing safety, performance, and storage efficiency.