Course contentsShow
← Machine Learning and Deep Learning
Lesson 1685 of 3,53836. LLM Inference OptimizationFree lesson

Multi-Query Attention

Extreme KV sharing where all query heads use the same key-value heads, maximizing memory efficiency.

Multi-Query Attention

What you'll learn: How extreme KV sharing—where all query heads use the same single key-value head—maximizes memory efficiency in transformer inference.

The Core Idea

In standard multi-head attention, each head maintains its own query, key, and value projections. You've seen Grouped-Query Attention reduce memory by sharing key-value heads across groups of query heads. Multi-Query Attention (MQA) takes this to the extreme: all query heads share a single key-value head.

Think of it like a library with multiple readers (query heads) but only one catalog system (key-value pair). Every reader uses the same catalog to find information, dramatically reducing storage requirements.

How It Works

Instead of projecting keys and values H times (once per head), MQA:

  1. Projects keys and values once to create a single shared KV head
  2. Projects queries H times as usual (one per attention head)
  3. Each query head attends to the same shared key-value representations

During the KV cache phase, this means storing only 1/H of the normal key-value cache, where H is the number of attention heads. For a 32-head model, that's a 32× reduction in KV cache memory!

The Trade-off

MQA sacrifices some representational capacity—multiple heads can't specialize with different key-value patterns anymore. However, empirical results show surprisingly small quality degradation (often <1% on benchmarks) while delivering massive inference speedups, especially for long sequences where KV cache dominates memory usage.

This makes MQA particularly valuable for production deployments where you need to serve many concurrent users or handle very long contexts within memory constraints.

Key Takeaway: Multi-Query Attention maximizes memory efficiency by sharing a single key-value head across all query heads, reducing KV cache size by a factor equal to the number of heads with minimal quality loss.