In this lesson, you will learn how to bypass the overhead of heap allocation by packing small data structures directly into the memory footprint of an enum or a fixed-size buffer.
Why Avoid the Heap?
When you store small pieces of data—like a short string or a tiny vector—in a standard Vec<T> or String, Rust allocates space on the heap. This introduces two performance killers: an extra pointer dereference (which risks cache misses) and the overhead of managing the heap allocator. Small-Object Optimization (SOO) allows us to keep that data "inline" when it is small enough, effectively keeping our working set within the CPU cache.
The "Inline or Allocate" Pattern
The most common way to implement SOO in Rust is by using an enum that distinguishes between a stack-allocated "fast path" and a heap-allocated "overflow path."
Consider an identifier structure that usually holds 8 bytes but occasionally needs more:
enum CompactId {
Inline([u8; 8]),
Heap(Box<[u8]>),
}
By keeping the data inside the enum variant itself, the memory layout remains contiguous. When the program encounters an Inline variant, it accesses the bytes directly from the stack, avoiding the cache-draining trip to the heap.
Balancing Memory Footprint
While SOO improves cache locality, it increases the size of your objects. Every instance of CompactId must now be large enough to hold the Inline buffer plus the enum tag and the Box pointer. If you set your Inline buffer too large, you might inflate the size of your collection entries, potentially reducing the number of elements that fit in a single L1 cache line.
The goal is to choose a threshold that covers the "common case"—the statistical majority of your data—while offloading the long-tail exceptions to the heap. By aligning your data size with typical CPU cache line boundaries, you turn frequent memory-bound operations into high-speed local data accesses.
Key Takeaway: Small-Object Optimization boosts performance by sacrificing a small amount of fixed stack space to eliminate heap allocation for common, tiny data instances, thereby keeping your data closer to the CPU.