The Cost of Indirection in Generics

In this lesson, you will learn how the way you handle generic types affects how your CPU accesses data in memory.

When you use generics in Rust, the compiler generally uses static dispatch (monomorphization). It creates a unique copy of your function for every concrete type you use. Because the compiler knows the exact type at compile time, it generates code that jumps directly to the memory address of the data. This is fast, predictable, and allows the compiler to "inline" the function code, removing the overhead of function calls entirely.

However, when we introduce indirection—specifically via dynamic dispatch (using trait objects like &dyn Trait)—the situation changes.

The Cost of a Vtable

When you use &dyn Trait, the compiler no longer knows the exact type. Instead, it places the data behind a pointer and pairs it with a vtable (a virtual method table). A vtable is essentially a lookup table of function pointers.

1. Pointer Dereference: To call a method, the CPU must first load the address of the vtable from the trait object.
2. Memory Jump: The CPU then looks up the specific function address in that table.
3. Indirect Call: Finally, the CPU jumps to that address.

// Static Dispatch: The compiler knows the type, no vtable needed.
fn run_static<T: Speak>(item: T) { item.speak(); }

// Dynamic Dispatch: Uses a vtable; involves pointer indirection.
fn run_dynamic(item: &dyn Speak) { item.speak(); }

Why it Matters for Performance

This "extra hop" through memory is called indirection. It creates two performance hurdles:

• Instruction Cache Misses: The CPU cannot easily predict which code will run next because the target address is hidden in memory, preventing the compiler from inlining the code.
• Data Locality: Because your data is behind a pointer, it might be scattered in a different part of the heap, causing "cache misses" where the CPU has to wait for data to be fetched from slower system memory rather than the lightning-fast CPU cache.

In high-performance systems, we favor static dispatch to keep data layout tight and predictable, using dynamic dispatch only when we need to group different types together at runtime.

Key Takeaway: Static dispatch allows the compiler to inline code and optimize memory access, whereas dynamic dispatch introduces indirection that hides the target code from the compiler and disrupts CPU cache efficiency.