Course contentsShow
← Machine Learning and Deep Learning
Lesson 2652 of 3,53857. Model Compression: QuantizationFree lesson

QAT in PyTorch

Using torch.quantization for QAT: prepare_qat, training with fake quant, and converting to quantized model.

QAT in PyTorch

What You'll Learn

You'll learn how to implement Quantization-Aware Training in PyTorch using the built-in torch.quantization module, covering model preparation, training with fake quantization, and final conversion.

Core Idea

PyTorch provides a straightforward workflow for QAT through three main steps: prepare, train, and convert. Instead of manually inserting fake quantization nodes, PyTorch automates this process while giving you control over which layers get quantized and how.

The Three-Step QAT Workflow

1. Prepare the Model with prepare_qat

First, configure your model with a QConfig that specifies quantization settings (like per-channel or per-tensor), then call prepare_qat. This function inserts fake quantization nodes for both weights and activations, essentially creating a "quantization-aware" version of your model.

import torch.quantization as quantization

# Configure quantization
model.qconfig = quantization.get_default_qat_qconfig('fbgemm')

# Insert fake quant nodes
model_prepared = quantization.prepare_qat(model, inplace=False)

2. Train with Fake Quantization

Train your prepared model normally. During forward passes, fake quantization simulates quantization effects (rounding, clipping) while gradients flow through using the straight-through estimator during backpropagation. The model learns to be robust to quantization noise.

3. Convert to Quantized Model

After training, call convert to replace fake quantization operations with actual integer ops:

model_quantized = quantization.convert(model_prepared, inplace=False)

Now your model runs with real INT8 operations, delivering actual speedup and size reduction.

Analogy

Think of prepare_qat as putting training wheels on a bicycle (fake quant), training as learning to balance with those wheels on, and convert as removing the training wheels—you're now riding with the real thing.

Key Takeaway: PyTorch QAT uses prepare_qat to insert fake quantization, trains the model to adapt to quantization constraints, then convert produces the final INT8 model ready for efficient deployment.