Optimizing Data Locality

In this lesson, you will learn how to restructure your data layouts to align with CPU cache behavior, drastically reducing latency by maximizing spatial locality.

Understanding the Cache Line

Modern CPUs don't fetch single bytes from RAM; they fetch "cache lines"—typically 64 bytes at a time. If your code accesses a piece of data, the hardware loads that entire chunk into the L1 cache. If your data is laid out poorly, the CPU will frequently fetch data it doesn't need, wasting bandwidth and triggering costly cache misses.

Struct of Arrays (SoA) vs. Array of Structs (AoS)

The standard approach in Rust is Array of Structs (AoS):

struct Particle {
    x: f32, y: f32, z: f32, // 12 bytes
    health: f32,            // 4 bytes
    active: bool,           // 1 byte
}

If you are running a simulation that only updates the position (x, y, z), the CPU is forced to load health and active into the cache for every single particle because they occupy the same cache line. This is inefficient if health isn't needed in that specific loop.

By switching to a Struct of Arrays (SoA), you group similar data types together:

struct ParticleSystem {
    x: Vec<f32>,
    y: Vec<f32>,
    z: Vec<f32>,
    health: Vec<f32>,
}

Now, when you iterate over x, the cache line is packed exclusively with the next sequence of x coordinates. Your traversal becomes perfectly linear, and the hardware prefetcher can accurately predict the next memory address, keeping the CPU pipeline fed with data.

Practical Application

To optimize locality, inspect your "hot loops." If you find yourself frequently accessing only a subset of fields within a struct, consider refactoring your storage. Using #[repr(align(64))] can also prevent "false sharing," where unrelated data objects inhabit the same cache line and cause unnecessary synchronization overhead during multi-threaded access.

By reorganizing data to match your access patterns, you stop fighting the memory hierarchy and start working with it, ensuring that every byte brought into the cache is put to immediate work.

Key Takeaway: To maximize performance, reorganize your memory layout so that data accessed together is stored contiguously, ensuring each cache line fetch contains only the information your loop requires.