Calibration Algorithms: MinMax and Percentile
What You'll Learn
You'll learn how simple calibration methods use minimum/maximum values or percentiles to compute quantization ranges from activation distributions.
The Core Problem
When performing static quantization, you need to determine the dynamic range of activations before inference begins. This range (min and max values) determines your scale and zero-point parameters. But how do you find these values without seeing actual inference data?
The answer: calibration — running representative data through your model and observing activation distributions.
MinMax Calibration
The simplest approach: record the absolute minimum and maximum values seen during calibration.
How it works:
- Pass calibration data through your model
- For each layer, track
min_valandmax_valof activations - Use these extremes as your quantization range
Analogy: Like adjusting a thermostat based on the coldest and hottest days of the year.
Drawback: Sensitive to outliers. One extreme value can waste your quantization resolution.
Percentile Calibration
A more robust approach: use percentiles (e.g., 0.1% and 99.9%) instead of true extremes.
How it works:
- Collect activation values during calibration
- Compute the Pth percentile (e.g., 1st) and (100-P)th percentile (e.g., 99th)
- Use these as your min/max range
Analogy: Like setting temperature based on "typical" cold/hot days, ignoring rare extremes.
Advantage: Clips outliers, preserving quantization precision for the bulk of your data.
Choosing Between Them
- MinMax: Simple, fast, works when data is well-behaved
- Percentile: Better for distributions with outliers, requires storing more calibration statistics
Key Takeaway: MinMax uses true extremes for quantization ranges, while percentile methods clip outliers at specified percentiles (e.g., 99.9%) to create more robust ranges that preserve precision for typical values.