Calibration for Static Quantization
What you'll learn: How to use representative data to determine the best quantization parameters before deployment.
Why Calibration Matters
In static quantization, you quantize your model before inference time. But here's the problem: how do you know what range your activations will take? Unlike weights (which are fixed), activations depend on input data.
Calibration solves this by running your model on a small, representative dataset to observe the actual activation ranges. Think of it like a dress rehearsal before the real performance—you're measuring what actually happens so you can set optimal quantization parameters.
How Calibration Works
During calibration, you:
- Collect statistics: Pass representative data through your model and record the min/max (or percentile-based ranges) of each activation layer
- Compute scale and zero-point: Use the observed ranges to calculate quantization parameters
- Lock them in: These parameters become fixed for all future inference
For example, if an activation layer consistently outputs values between -2.5 and 3.8 during calibration, you'll use that range to compute the scale and zero_point that map those floats to your target integer range (like 0-255 for 8-bit).
Choosing Calibration Data
Your calibration dataset should be representative but doesn't need to be huge—typically 100-1000 samples work well. It should cover the diversity of inputs your model will see in production. Poor calibration data leads to clipping (values outside the observed range) and accuracy loss.
Calibration Methods
Common approaches include:
- Min-Max: Use the absolute minimum and maximum observed values
- Percentile-based: Use 99th percentile to ignore outliers (more robust)
- Entropy minimization: Choose ranges that minimize information loss
Key Takeaway: Calibration runs your model on representative data to measure actual activation ranges, allowing you to set optimal quantization parameters for static quantization before deployment.