Evaluating PTQ Accuracy Degradation
What you'll learn: How to measure accuracy loss from post-training quantization, identify which layers cause problems, and determine when you need advanced techniques.
Why Evaluation Matters
After applying post-training quantization (PTQ), your model is smaller and faster—but at what cost? Not all models tolerate quantization equally well. Some lose minimal accuracy, while others degrade significantly. Your job is to measure the damage and diagnose the cause.
Measuring Accuracy Loss
Compare your quantized model's performance against the original floating-point baseline using the same test set:
- Absolute degradation:
original_accuracy - quantized_accuracy - Relative degradation:
(original_accuracy - quantized_accuracy) / original_accuracy × 100%
For example, if your model drops from 92% to 89% accuracy, that's 3 percentage points absolute or ~3.26% relative loss.
Identifying Problematic Layers
Not all layers quantize gracefully. Use per-layer sensitivity analysis: quantize layers individually and measure accuracy drop. Layers with wide dynamic ranges (large min/max weight values) or outlier activations often cause the most degradation.
Think of it like compressing images—smooth gradients compress well, but fine details (high-frequency components) suffer. Similarly, layers with subtle weight distributions lose precision.
When to Escalate
If PTQ accuracy loss exceeds your tolerance (typically >1-2% for most applications):
- Try per-channel quantization for sensitive layers
- Use mixed-precision: keep problematic layers in FP16/FP32
- Switch to Quantization-Aware Training (QAT) to recover accuracy through fine-tuning
Key Takeaway: Evaluating PTQ accuracy degradation means comparing quantized vs original performance, profiling per-layer sensitivity, and deciding whether simple PTQ suffices or if you need mixed-precision or QAT intervention.