Course contentsShow
← System Design
Lesson 714 of 1,91919. Event Streaming PlatformsFree lesson

Kafka's Read Path and Zero-Copy

Understand how consumers fetch data efficiently using sequential reads and zero-copy transfers from disk to network.

Kafka's Read Path and Zero-Copy

You'll learn: How Kafka consumers fetch messages blazingly fast by reading sequentially from disk and skipping unnecessary data copies.

The Read Path Journey

When a consumer requests messages from Kafka, the broker must locate the right log segment on disk, read the data, and send it over the network. The naive approach would copy data multiple times: disk → kernel buffer → application buffer → socket buffer → network card. Each copy wastes CPU and memory.

Kafka avoids this overhead through sequential disk reads and zero-copy transfer.

Sequential Reads: Leveraging the Commit Log

Because Kafka stores messages in an append-only commit log (ordered by offset), consumers typically read forward sequentially. Sequential disk access is fast — modern disks can sustain hundreds of MB/sec for sequential reads, almost as fast as network speeds. The operating system's page cache also helps: recently written data is often still in memory when consumers read it shortly after.

Analogy: Reading a book page-by-page (sequential) is much faster than flipping randomly to different chapters (random access).

Zero-Copy: Direct Disk-to-Network

Kafka uses the sendfile() system call (Linux) to transfer data directly from the disk file to the network socket, bypassing application memory entirely. The kernel handles everything internally. This eliminates multiple buffer copies, reduces CPU usage, and increases throughput dramatically.

Instead of: Disk → Kernel → App → Kernel → Network
Zero-copy does: Disk → Kernel → Network

The broker doesn't even "look at" the message bytes during this transfer — it just moves chunks of the log file straight to the consumer's TCP connection.

Key Takeaway:
Kafka's read path achieves high throughput by exploiting sequential disk access patterns and zero-copy transfers, allowing brokers to serve consumers with minimal CPU overhead and latency.