Course contentsShow
← Machine Learning and Deep Learning
Lesson 2643 of 3,53857. Model Compression: QuantizationFree lesson

Quantization-Aware Training: Motivation and Overview

Why QAT outperforms post-training quantization and how simulating quantization during training preserves accuracy.

Quantization-Aware Training: Motivation and Overview

You'll learn why training with quantization in mind beats quantizing after training and how it works.

Why Post-Training Quantization Falls Short

When you quantize a model after training (PTQ), the weights and activations are forced into lower precision bins they were never optimized for. This can cause significant accuracy degradation because the model learned fine-grained floating-point patterns that get disrupted by rounding errors.

Think of it like this: imagine learning to play piano with full finger control, then suddenly being asked to perform wearing thick gloves. Your muscle memory doesn't account for the reduced precision.

The QAT Solution

Quantization-Aware Training (QAT) simulates quantization during training itself. The forward pass uses quantized (low-precision) values, but gradients still flow through in full precision during backpropagation. This way, the model learns weight values and activation patterns that are robust to quantization from the start.

The network effectively learns to work within the constraints of reduced precision. It discovers weight configurations where small rounding errors don't catastrophically affect predictions.

How It Works

During training, QAT inserts "fake quantization" operations:

  1. Convert weights/activations to low precision (simulate quantization)
  2. Immediately convert back to floating point (dequantize)
  3. Continue forward pass with these dequantized values
  4. Backpropagate gradients through these operations using straight-through estimators

The model gradually adapts its weights to minimize loss under quantized conditions, not ideal floating-point conditions.

Why It Outperforms PTQ

QAT typically recovers 90-100% of the original accuracy even with INT8 quantization, while PTQ might lose 2-5% or more. The model has thousands of gradient updates to compensate for quantization noise, rather than a one-time post-hoc compression.

Key Takeaway: Quantization-Aware Training simulates low-precision operations during training so the model learns weights that remain accurate after quantization, dramatically outperforming post-training quantization approaches.