In this lesson, you will learn how to minimize memory overhead when your application requires millions of tiny, independent heap allocations.
The Hidden Cost of Small Allocations
When you create millions of small objects using Box<T> or Arc<T>, you aren't just paying for the size of T. Every heap allocation carries "allocation metadata"—the overhead the allocator uses to track the size and state of the block. On many systems, this can add 8 to 16 bytes per allocation. If your data structure is only 4 bytes, you are spending 200–400% more memory on metadata than on your actual data.
Furthermore, heap-allocated objects are scattered across memory, causing cache misses and increasing memory fragmentation.
Strategy: Data Flattening
Instead of storing a collection of pointers to individual objects, aim to store the objects contiguously in a single allocation. This is the foundation of Data-Oriented Design.
Consider this inefficient pattern:
// Inefficient: Millions of allocations
let items: Vec<Box<MyStruct>> = (0..1_000_000)
.map(|i| Box::new(MyStruct::new(i)))
.collect();
By switching to a flat Vec<T>, you eliminate the box overhead entirely and improve cache locality:
// Efficient: One large allocation
let items: Vec<MyStruct> = (0..1_000_000)
.map(|i| MyStruct::new(i))
.collect();
When You Need Indirection
If you must have pointers (e.g., for trait objects or shared state), consider using an Arena Allocator or a Slot Map. An arena allocates a large contiguous "slab" of memory upfront and carves it into smaller pieces. Instead of a Box<T>, you store a small index or an integer handle. This drastically reduces the pressure on the global allocator, as the system sees one massive allocation rather than millions of micro-allocations.
By keeping data contiguous, you reduce the memory footprint by stripping away pointer overhead and metadata, while simultaneously making your data structures friendlier to the CPU's prefetcher.
Key Takeaway: Never allocate a million small objects when you can allocate one large, contiguous buffer and index into it.