Memory Alignment and Performance

In this lesson, you will learn how CPU cache efficiency is dictated by the alignment of your data structures in memory.

The Cache Line Reality

To your CPU, memory isn't just a giant list of bytes; it is organized into fixed-size chunks called cache lines (typically 64 bytes). When the CPU needs to read a single u32 (4 bytes), it doesn't just grab those four bytes. It fetches the entire 64-byte cache line containing that data and loads it into the high-speed cache.

Performance problems arise when data structures are "misaligned" or scattered across cache boundaries.

The Alignment Tax

If a piece of data is placed such that it straddles two cache lines, the CPU must perform two memory fetches instead of one to read a single value. Furthermore, if your data layout is sparse, you might fill a cache line with "padding" bytes—wasted space that isn't actually storing useful information, but is taking up limited room in your cache.

Consider this Rust structure:

struct BadLayout {
    a: u8,   // 1 byte
    // 7 bytes of implicit padding added here by the compiler
    b: u64,  // 8 bytes
}

In the example above, the compiler inserts padding after a to ensure b starts at an address divisible by 8 (its size). This ensures the CPU can access b efficiently. However, if you had many such structures in a list, you would have significant "dead space" that occupies your cache, forcing the CPU to fetch new cache lines more frequently than if the data were packed tightly.

Optimizing for the Cache

To maximize performance, we arrange fields within our structs to minimize padding. By grouping larger types together and placing smaller types at the end, we help the compiler pack data into the smallest footprint possible. When your data is perfectly aligned and packed, more "useful" data fits into every 64-byte fetch. This minimizes cache misses and keeps your CPU working at its maximum potential, rather than waiting for slow memory loads.

Key Takeaway: Aligning your data structures minimizes padding and ensures that each cache line fetch brings the maximum amount of useful information into the CPU, directly reducing latency.