Exploiting Niche Optimizations

In this lesson, you will learn how to design data types that take advantage of Rust’s "niche" optimization to reduce memory usage without sacrificing type safety.

What is a Niche?

In Rust, a "niche" is a bit pattern in a type’s memory representation that is logically impossible or unused. When the compiler notices a niche, it can store extra information inside those unused bits. The most famous example is the Option<T> type.

Normally, if you have an Option<T>, you might expect it to take the size of T plus an extra byte for the "tag" (to tell the compiler if it is Some or None). However, if T is a reference or a Box, the compiler knows that a null pointer is an invalid state for that type. Rust uses that "null" value as the tag for None, meaning Option<&T> has the exact same memory footprint as a raw &T.

Designing for Niches

You can exploit this behavior when designing your own custom types. Consider an enum representing a state:

enum State {
    Active,    // 0
    Paused,    // 1
    Finished,  // 2
}

This enum requires a full byte. But if you wrap this in an Option<State>, it will jump to 2 bytes: one for the State and one for the Option tag.

If you want to save that extra byte, you can define a niche by using a type that doesn't use all its bit patterns:

#[repr(u8)]
enum State {
    Active = 1,
    Paused = 2,
    Finished = 3,
    // 0 is unused, creating a niche!
}

Because 0 is not a valid discriminant for your enum, the compiler can use 0 to represent None if you wrap it in Option<State>. This allows the Option to fit into the same space as the enum itself. By explicitly setting your discriminants to leave a gap, you enable the compiler to perform "null pointer optimization" on your own custom types.

Always keep an eye on your struct sizes using std::mem::size_of. If you see an Option wrapping a small type that is larger than the type itself, check if your discriminants or fields provide a bit pattern the compiler can reuse as a niche.

Key Takeaway: By leaving specific bit patterns unused in your types, you allow the compiler to use those "niches" to store state information, effectively reducing memory overhead to zero-cost.