PyTorch Static Quantization with QConfig
What You'll Learn
You'll learn how to configure and apply static quantization in PyTorch using QConfig objects to control quantization behavior for different layers.
The Core Idea
Static quantization requires calibration (running representative data through your model) to determine optimal quantization parameters. PyTorch uses QConfig objects to specify how different layers should be quantized—defining both the observer for activation statistics and the fake-quantization behavior during calibration.
Think of QConfig as a "recipe card" you attach to layers: it tells PyTorch "use this method to observe activations, and this method for weights." Different layers may need different recipes (e.g., convolutions vs linear layers).
How It Works
- Create a QConfig: Combine an activation observer and weight observer
- Attach to model: Use
qconfigto specify quantization behavior - Prepare: Insert observers into the model
- Calibrate: Run sample data to collect statistics
- Convert: Replace operations with quantized versions
import torch
import torch.quantization as quant
# Define a QConfig (activation + weight observers)
qconfig = quant.QConfig(
activation=quant.observer.MinMaxObserver,
weight=quant.observer.MinMaxObserver
)
# Attach to model
model.qconfig = qconfig
# Prepare for calibration
model_prepared = quant.prepare(model, inplace=False)
# Calibrate with sample data
for data in calibration_loader:
model_prepared(data)
# Convert to quantized model
model_quantized = quant.convert(model_prepared, inplace=False)
Different QConfig presets exist: get_default_qconfig(), get_default_qat_qconfig(), or custom combinations using different observers from previous lessons.
Key Takeaway: QConfig objects in PyTorch let you precisely control static quantization by specifying which observers track activation and weight statistics during calibration, enabling fine-grained quantization strategies per layer.