AWQ: Activation-Aware Weight Quantization
What You'll Learn
You'll learn how AWQ protects the most important weights in large language models during quantization by analyzing activation magnitudes, achieving better performance than uniform quantization approaches.
The Core Problem
When you quantize all weights uniformly (like GPTQ does), you treat every parameter equally. But not all weights contribute equally to model quality. Some weights interact with large activation values and have outsized impact on the final output—quantizing these "salient" weights aggressively causes significant accuracy loss.
How AWQ Works
AWQ takes an activation-aware approach: it analyzes which weights multiply with the largest activations during inference. Here's the insight:
Output error = quantization_error × activation_magnitude
A small quantization error on a weight that multiplies a huge activation creates massive output error. Conversely, quantizing a weight that pairs with tiny activations has minimal impact.
The AWQ Strategy:
- Profile activations on calibration data to find their magnitudes
- Identify salient weights that consistently interact with large activations
- Protect these weights by keeping them at higher precision or applying minimal quantization
- Aggressively quantize less-important weights to maintain overall compression
Think of it like protecting star players on a sports team—you can't afford to handicap your MVPs, but substitutes can take more cuts without hurting overall performance.
The Advantage
Unlike GPTQ's layer-by-layer reconstruction, AWQ makes global decisions based on what actually matters during inference. This typically yields better accuracy at the same bit-width (e.g., INT4) for large language models, especially on tasks requiring precise reasoning.
Key Takeaway: AWQ achieves superior quantized LLM performance by protecting salient weights—those that multiply large activations—while aggressively compressing less critical parameters, making smarter trade-offs than uniform quantization.