Block Management and Page Tables
What you'll learn: How vLLM tracks which pieces of KV cache data live where in GPU memory using page tables—just like your operating system manages RAM.
The Core Idea
When vLLM uses Paged Attention, it breaks each sequence's KV cache into fixed-size blocks (the "pages"). But these blocks don't sit in contiguous memory—they scatter across available GPU memory wherever space exists. To keep track of this fragmentation, vLLM maintains a page table for each sequence.
Think of it like a library catalog: the books (KV cache blocks) can be stored anywhere on the shelves (physical GPU memory), but the catalog (page table) tells you exactly where to find each chapter of each book.
How Page Tables Work
Each sequence has its own page table—a mapping data structure that translates logical block numbers to physical memory addresses:
- Logical blocks: Sequential indices (0, 1, 2, ...) representing the conceptual order of KV cache blocks for a sequence
- Physical blocks: Actual GPU memory locations where those blocks are stored
When the attention mechanism needs token position 457's keys and values, vLLM:
- Calculates which logical block contains position 457 (e.g., block 22)
- Looks up block 22 in the sequence's page table
- Finds the physical memory address (e.g., GPU memory slot 891)
- Reads the data from that physical location
This indirection allows sequences to share physical blocks (for prompt prefixes), grow non-contiguously, and be swapped in/out of memory efficiently.
Key Takeaway:
vLLM's page tables map each sequence's logical KV cache block indices to scattered physical GPU memory locations, enabling flexible memory allocation and sharing without requiring contiguous storage.