In this lesson, you will learn how the transition from stack-allocated data to heap-allocated pointers impacts CPU cache efficiency and introduces latency.
The Cost of Indirection
When you store data directly on the stack, the CPU knows exactly where that data lives relative to the stack pointer. Accessing it is a direct operation. However, when you move data to the heap—using Box<T> or other smart pointers—you introduce indirection.
Indirection forces the CPU to perform a "pointer chase." To read your data, the CPU must first load the address stored in the pointer from the stack, and then perform a second memory fetch to retrieve the actual value from the heap.
Cache Locality and The Memory Wall
The real performance penalty isn't just the extra instruction to dereference the pointer; it is the destruction of spatial locality.
Consider a Vec of i32 versus a Vec of Box<i32>:
// Contiguous: Values are packed tightly, easy for the CPU prefetcher.
let stack_vec: Vec<i32> = vec![1, 2, 3, 4];
// Indirection: The Vec stores pointers, and the integers are scattered
// across the heap. Each access requires a jump to a random memory location.
let heap_vec: Vec<Box<i32>> = vec![Box::new(1), Box::new(2), Box::new(3), Box::new(4)];
When you iterate over stack_vec, the CPU prefetcher sees a predictable pattern and loads a "cache line" (typically 64 bytes) containing multiple integers at once. Because they are adjacent, the next integer is likely already in the L1 cache before you even ask for it.
In contrast, when you iterate over heap_vec, the pointers may point to memory addresses that are far apart. Each dereference risks a cache miss, forcing the CPU to stall while it fetches data from slower main memory (RAM). This "pointer chasing" creates a pipeline stall, as the CPU cannot calculate the address of the next item until the previous memory fetch completes.
In high-performance Rust, we prefer keeping data contiguous to maximize cache line utilization, reserving indirection only when the structural requirements of the program (like recursive types) demand it.
Key Takeaway: Indirection replaces predictable, high-speed sequential memory access with unpredictable pointer chasing, which degrades performance by causing frequent CPU cache misses.