Course contentsShow
← AI Engineering
Lesson 1034 of 1,88625. Model Serving and Inference OptimizationFree lesson

Grouped-Query Attention (GQA)

GQA as a middle ground between multi-head and multi-query attention for memory-quality balance.

Grouped-Query Attention (GQA)

What You'll Learn

You'll understand how Grouped-Query Attention offers a balanced compromise between Multi-Head Attention and Multi-Query Attention for efficient LLM serving.

The Middle Ground

After learning about Multi-Query Attention (MQA), which uses a single key-value pair shared across all query heads, you might wonder: is there something between full multi-head attention and aggressive sharing?

Grouped-Query Attention (GQA) is exactly that middle ground. Instead of having one KV pair for all queries (MQA) or one KV pair per query head (standard multi-head), GQA organizes query heads into groups that share KV pairs.

How It Works

Imagine a model with 32 query heads:

  • Multi-Head Attention: 32 query heads, 32 KV pairs → maximum quality, maximum memory
  • Multi-Query Attention: 32 query heads, 1 KV pair → minimum memory, potential quality loss
  • Grouped-Query Attention: 32 query heads, 4 KV pairs → 8 heads share each KV pair

The Sweet Spot

GQA typically groups 4-8 query heads per KV pair. This configuration:

  • Reduces KV cache memory by 4-8× compared to full multi-head attention
  • Preserves more model quality than MQA by maintaining multiple KV representations
  • Increases throughput through smaller cache footprint and faster memory operations
  • Maintains flexibility by allowing you to tune the group size based on your memory/quality requirements

Think of it like organizing a restaurant kitchen: instead of one chef per dish (multi-head) or one chef for everything (MQA), you have specialized stations where small teams share equipment efficiently.

Key Takeaway: Grouped-Query Attention provides a tunable balance between memory efficiency and model quality by grouping query heads to share KV pairs, making it an increasingly popular choice for production LLM deployment.