Post-Training Quantization (PTQ)
What You'll Learn
You'll learn how to compress already-trained models by reducing their numeric precision using calibration data—no retraining required.
The Core Idea
Post-Training Quantization (PTQ) is the process of converting a pre-trained model from higher precision (like FP32 or FP16) to lower precision formats (INT8 or INT4) after training is complete. Think of it like compressing a high-resolution photo: you lose some fine detail, but the image remains recognizable and takes up much less space.
The key to PTQ is using a calibration dataset—a small, representative sample of real inputs. This dataset helps determine the optimal scaling factors and ranges for converting floating-point weights and activations to integers without destroying model accuracy.
How It Works
- Load your pre-trained model in its original precision (FP32/FP16)
- Run the calibration dataset through the model to observe activation distributions
- Calculate optimal quantization parameters (scale and zero-point values) for each layer
- Convert weights and activations to lower precision (INT8/INT4)
- Validate accuracy on your test set
The calibration step is crucial—it ensures that your quantized model preserves the statistical properties of the original, minimizing accuracy loss. You typically need only 100-1000 calibration samples, not the full training set.
Why PTQ?
Unlike Quantization-Aware Training (QAT), PTQ requires no gradient updates or backpropagation. It's faster to implement, doesn't need training infrastructure, and works with models you don't have training code for. The tradeoff? PTQ sometimes loses slightly more accuracy than QAT, but modern techniques have narrowed this gap considerably.
Key Takeaway: Post-Training Quantization compresses pre-trained models to lower precision using calibration data to determine optimal conversion parameters—no retraining needed, making it the fastest path to deployment optimization.