In this lesson, you will learn how to align your data structures with CPU cache lines to minimize latency and maximize memory throughput.
The Cache-Aware Reality
Modern processors do not fetch data from RAM byte-by-byte; they fetch "cache lines"—usually 64 bytes at a time. If your data is scattered across the heap (as is common with pointer-heavy object-oriented patterns), the CPU constantly stalls while waiting for memory fetches. Data-Oriented Design (DOD) flips this by organizing memory so that the data needed for a computation is contiguous, ensuring that when the CPU loads one item, it automatically pulls the next several items into the cache simultaneously.
Aligning for Throughput
In Rust, you can control the physical footprint of your structures using the #[repr(C)] attribute combined with #[repr(align(N))]. This forces the compiler to respect specific memory boundaries, preventing the CPU from needing multiple cache line accesses to read a single object.
Consider a system processing thousands of particle updates:
#[repr(C)]
#[repr(align(64))]
struct Particle {
pos: [f32; 3], // 12 bytes
vel: [f32; 3], // 12 bytes
_pad: [u8; 40], // Padding to fill the 64-byte cache line
}
By ensuring each Particle is exactly 64 bytes and starts at the beginning of a cache line, you eliminate "false sharing" and ensure that traversing an array of Particle structs is a linear, predictable operation. When the CPU fetches particles[0], the hardware prefetcher is highly likely to load particles[1] before you even ask for it, effectively hiding memory latency.
The DOD Mindset
Data-Oriented Design shifts the focus from "what is this object?" to "how is this data accessed?" If you have a collection of components, don't store them in a Vec<Object>. Store them in a Vec<ComponentA> and a Vec<ComponentB>. This "Structure of Arrays" (SoA) approach ensures that when you iterate over just the positions, your cache is filled exclusively with coordinate data, not irrelevant metadata like pointers or flags.
Key Takeaway: Design your data structures to fit into power-of-two cache lines, keeping frequently accessed fields contiguous to ensure the CPU hardware prefetcher can stream data into the cache without stalls.