INT8 Calibration in TensorRT
What you'll learn: How TensorRT uses calibration datasets to find optimal INT8 quantization scales that preserve model accuracy while maximizing performance.
The Calibration Challenge
When you quantize a model from FP32 to INT8, you're compressing values into a much narrower range (256 discrete values instead of billions). The key question is: which 256 values should represent your original distribution? Choose poorly, and important activations get clipped or rounded away. Choose well, and you keep nearly all the model's accuracy while gaining 4x memory reduction and dramatic speedup.
How INT8 Calibration Works
Think of calibration like measuring a room before buying furniture. You need representative measurements to make good decisions.
TensorRT's calibration process:
- Run inference on a small, representative calibration dataset (typically 500-1000 samples from your training or validation set)
- Collect activation histograms at each layer to understand the distribution of values
- Compute optimal scales that minimize information loss—typically using entropy minimization (KL divergence) or percentile methods
- Generate a calibration cache storing these scales for each tensor
The calibration dataset must represent real-world input distributions. If you calibrate on cat images but deploy on dog images with different intensity ranges, your scales will be suboptimal.
Calibration Methods
TensorRT offers calibration algorithms:
- Entropy calibration: Minimizes information loss between FP32 and INT8 distributions
- MinMax: Simple but can be sensitive to outliers
- Percentile: Clips extreme values to avoid wasting quantization range
Key Takeaway: INT8 calibration in TensorRT automatically determines per-layer quantization scales by analyzing activation distributions on a representative dataset, enabling near-FP32 accuracy at 4x the speed with minimal manual tuning.