Course contentsShow
← Machine Learning and Deep Learning
Lesson 2664 of 3,53857. Model Compression: QuantizationFree lesson

AWQ: Activation-Aware Weight Quantization

Protecting salient weights based on activation magnitudes for better quantized LLM performance.

AWQ: Activation-Aware Weight Quantization

What You'll Learn

You'll learn how AWQ protects the most important weights in large language models during quantization by analyzing activation magnitudes, achieving better performance than uniform quantization approaches.

The Core Problem

When you quantize all weights uniformly (like GPTQ does), you treat every parameter equally. But not all weights contribute equally to model quality. Some weights interact with large activation values and have outsized impact on the final output—quantizing these "salient" weights aggressively causes significant accuracy loss.

How AWQ Works

AWQ takes an activation-aware approach: it analyzes which weights multiply with the largest activations during inference. Here's the insight:

Output error = quantization_error × activation_magnitude

A small quantization error on a weight that multiplies a huge activation creates massive output error. Conversely, quantizing a weight that pairs with tiny activations has minimal impact.

The AWQ Strategy:

  1. Profile activations on calibration data to find their magnitudes
  2. Identify salient weights that consistently interact with large activations
  3. Protect these weights by keeping them at higher precision or applying minimal quantization
  4. Aggressively quantize less-important weights to maintain overall compression

Think of it like protecting star players on a sports team—you can't afford to handicap your MVPs, but substitutes can take more cuts without hurting overall performance.

The Advantage

Unlike GPTQ's layer-by-layer reconstruction, AWQ makes global decisions based on what actually matters during inference. This typically yields better accuracy at the same bit-width (e.g., INT4) for large language models, especially on tasks requiring precise reasoning.

Key Takeaway: AWQ achieves superior quantized LLM performance by protecting salient weights—those that multiply large activations—while aggressively compressing less critical parameters, making smarter trade-offs than uniform quantization.