Memory Layout and Locality of Reference

In this lesson, you will learn how to structure your Rust data to align with CPU cache lines, minimizing latency through spatial locality.

The Physics of Performance

Modern CPUs do not fetch single bytes from RAM. Instead, they load chunks of memory—typically 64 bytes—into a cache line. If your data is contiguous in memory, the CPU can prefetch the next items before your code even requests them. If your data is scattered (like a Vec<Box<T>>), the CPU suffers a "cache miss," forcing it to stall while waiting for the significantly slower main memory.

Structuring for Locality

When iterating over collections, the arrangement of your data dictates whether you are working with or against the hardware. Consider the difference between an Array of Structures (AoS) and a Structure of Arrays (SoA).

AoS (Potentially Cache-Unfriendly):

struct Particle {
    x: f32,
    y: f32,
    active: bool,
}
let particles: Vec<Particle> = vec![...];

If you only want to update the x coordinates of all particles, the CPU must load the entire Particle struct (including y and active) into the cache, effectively wasting cache space with data you aren't currently using.

SoA (Cache-Optimized):

struct ParticleSystem {
    x: Vec<f32>,
    y: Vec<f32>,
    active: Vec<bool>,
}

In the SoA pattern, the x values are packed tightly together. When you iterate over x, the cache line is filled entirely with f32 coordinates, ensuring that every byte loaded into the cache is utilized. This allows the CPU’s hardware prefetcher to identify the pattern and load upcoming memory segments ahead of time, maximizing the throughput of your iterator pipelines.

Data Alignment

Rust allows you to use #[repr(C)] or #[repr(align(n))] to control exactly how fields are padded and aligned in memory. By keeping hot data—data you access together—grouped within the same 64-byte boundary, you ensure that a single fetch captures the entire working set for that operation.

Key Takeaway: To maximize cache efficiency, store related data contiguously and prefer data layouts that allow your loops to traverse predictable, packed memory buffers.