Benchmarking with Criterion

In this lesson, you will learn how to use the Criterion library to generate precise, statistically significant measurements of your Rust code's performance.

Beyond Simple Timing

While you could try to measure code performance using a basic stopwatch or the standard library's Instant, those methods are often misleading due to background OS tasks, CPU frequency scaling, and cache warming. Criterion solves this by running your code many times, analyzing the statistical distribution of the results, and filtering out "noise" to provide an accurate report of your code's true speed.

Setting Up a Benchmark

To get started, you add criterion to your dev-dependencies in Cargo.toml. Since benchmarks live outside your main application logic, we place them in a dedicated benches/ directory at the root of your project.

Here is what a basic benchmark file (benches/my_benchmark.rs) looks like:

use criterion::{criterion_group, criterion_main, Criterion};

fn my_logic() -> u64 {
    (0..1000).sum()
}

fn bench_logic(c: &mut Criterion) {
    c.bench_function("summing 1000 numbers", |b| b.iter(|| my_logic()));
}

criterion_group!(benches, bench_logic);
criterion_main!(benches);

How It Works

When you run cargo bench, Criterion performs several automated steps:

1. Warm-up: It runs your function repeatedly to ensure the CPU is running at a stable frequency and the instruction cache is prepared.
2. Measurement: It runs the function inside an iter loop, timing how long it takes to complete across thousands of iterations.
3. Analysis: It calculates the mean, median, and variance of the execution time.
4. Reporting: It produces detailed HTML reports showing you exactly how your code performs and whether any recent changes have caused a "regression" (a performance drop).

By treating performance as a rigorous statistical exercise rather than a one-off guess, you can confidently identify the exact lines of code that are creating bottlenecks in your project.

Key Takeaway: Criterion provides statistically sound performance measurements by automating warm-ups and noise analysis, turning vague guesses into actionable data.