Grouped-Query Attention (GQA)
What You'll Learn
You'll understand how Grouped-Query Attention offers a balanced compromise between Multi-Head Attention and Multi-Query Attention for efficient LLM serving.
The Middle Ground
After learning about Multi-Query Attention (MQA), which uses a single key-value pair shared across all query heads, you might wonder: is there something between full multi-head attention and aggressive sharing?
Grouped-Query Attention (GQA) is exactly that middle ground. Instead of having one KV pair for all queries (MQA) or one KV pair per query head (standard multi-head), GQA organizes query heads into groups that share KV pairs.
How It Works
Imagine a model with 32 query heads:
- Multi-Head Attention: 32 query heads, 32 KV pairs → maximum quality, maximum memory
- Multi-Query Attention: 32 query heads, 1 KV pair → minimum memory, potential quality loss
- Grouped-Query Attention: 32 query heads, 4 KV pairs → 8 heads share each KV pair
The Sweet Spot
GQA typically groups 4-8 query heads per KV pair. This configuration:
- Reduces KV cache memory by 4-8× compared to full multi-head attention
- Preserves more model quality than MQA by maintaining multiple KV representations
- Increases throughput through smaller cache footprint and faster memory operations
- Maintains flexibility by allowing you to tune the group size based on your memory/quality requirements
Think of it like organizing a restaurant kitchen: instead of one chef per dish (multi-head) or one chef for everything (MQA), you have specialized stations where small teams share equipment efficiently.
Key Takeaway: Grouped-Query Attention provides a tunable balance between memory efficiency and model quality by grouping query heads to share KV pairs, making it an increasingly popular choice for production LLM deployment.