The Small-String Optimization

In this lesson, you will learn how to bypass the overhead of heap allocation for short strings by storing them directly on the stack.

Why Heap Allocations Cost

Normally, when you create a String in Rust, it allocates memory on the heap. This is great for long strings, but for tiny ones—like a single character or a short ID—the cost of asking the operating system for heap space can be slower than the work you are actually trying to do with the data.

The Small-String Optimization (SSO)

The core idea is simple: if a string is smaller than a certain threshold (e.g., 16 bytes), we store it inside a fixed-size buffer on the stack. If it grows larger, we "promote" it to the heap.

In practice, you can implement a simple version of this using a struct that holds an enum or a union of a stack buffer and a heap pointer:

struct TinyString {
    buffer: [u8; 16],
    len: u8,
    // If len > 16, we would use a heap-allocated pointer here
}

By keeping the data on the stack, you eliminate the pointer indirection and the need for a memory allocator. This makes your program more cache-friendly because the string data resides exactly where the TinyString struct lives, rather than being a separate lookup elsewhere in memory.

When to use it

You should consider the Small-String Optimization when you have a high-frequency component that handles thousands of short, temporary strings. If your strings are almost always shorter than 16 characters, you can drastically reduce your application’s memory fragmentation and latency.

While you could implement this from scratch for specific use cases, many high-performance crates (like smol_str or kstring) provide robust, battle-tested types that handle the logic of switching between the stack and the heap automatically. This allows you to reap the benefits of String—the ability to hold any length—while enjoying the speed of a stack-based array for the vast majority of your data.

Key Takeaway: By using a fixed-size stack buffer for short data, you can avoid heap allocation overhead and improve cache locality, significantly boosting performance for small string processing.