Course contentsShow
← Machine Learning and Deep Learning
Lesson 2642 of 3,53857. Model Compression: QuantizationFree lesson

Evaluating PTQ Accuracy Degradation

Learn to measure accuracy loss from PTQ, identify problematic layers, and decide when mixed-precision or QAT is needed.

Evaluating PTQ Accuracy Degradation

What you'll learn: How to measure accuracy loss from post-training quantization, identify which layers cause problems, and determine when you need advanced techniques.

Why Evaluation Matters

After applying post-training quantization (PTQ), your model is smaller and faster—but at what cost? Not all models tolerate quantization equally well. Some lose minimal accuracy, while others degrade significantly. Your job is to measure the damage and diagnose the cause.

Measuring Accuracy Loss

Compare your quantized model's performance against the original floating-point baseline using the same test set:

  • Absolute degradation: original_accuracy - quantized_accuracy
  • Relative degradation: (original_accuracy - quantized_accuracy) / original_accuracy × 100%

For example, if your model drops from 92% to 89% accuracy, that's 3 percentage points absolute or ~3.26% relative loss.

Identifying Problematic Layers

Not all layers quantize gracefully. Use per-layer sensitivity analysis: quantize layers individually and measure accuracy drop. Layers with wide dynamic ranges (large min/max weight values) or outlier activations often cause the most degradation.

Think of it like compressing images—smooth gradients compress well, but fine details (high-frequency components) suffer. Similarly, layers with subtle weight distributions lose precision.

When to Escalate

If PTQ accuracy loss exceeds your tolerance (typically >1-2% for most applications):

  1. Try per-channel quantization for sensitive layers
  2. Use mixed-precision: keep problematic layers in FP16/FP32
  3. Switch to Quantization-Aware Training (QAT) to recover accuracy through fine-tuning

Key Takeaway: Evaluating PTQ accuracy degradation means comparing quantized vs original performance, profiling per-layer sensitivity, and deciding whether simple PTQ suffices or if you need mixed-precision or QAT intervention.