Mean Reciprocal Rank (MRR)
You'll learn how to measure where your first relevant document appears in retrieval results using a single, intuitive metric.
What is MRR?
Mean Reciprocal Rank (MRR) tells you how good your RAG system is at putting relevant documents near the top of your search results. It focuses on a simple question: "How far down do users have to scroll before finding something useful?"
Think of it like searching for your car keys. If they're right on top of your desk (position 1), great! If they're buried under three books (position 4), that's less convenient. MRR quantifies this "convenience score" across many searches.
How It Works
For each query, find the rank of the first relevant document:
- If it's 1st → reciprocal rank = 1/1 = 1.0
- If it's 2nd → reciprocal rank = 1/2 = 0.5
- If it's 5th → reciprocal rank = 1/5 = 0.2
- If no relevant doc found → reciprocal rank = 0
Then average these reciprocal ranks across all queries. That's your MRR.
Example:
- Query 1: First relevant doc at position 1 → 1/1 = 1.0
- Query 2: First relevant doc at position 3 → 1/3 ≈ 0.33
- Query 3: First relevant doc at position 2 → 1/2 = 0.5
MRR = (1.0 + 0.33 + 0.5) / 3 ≈ 0.61
Why Use MRR?
MRR is perfect when you care most about that first click—common in question-answering systems where users expect the answer up top. Unlike precision and recall, MRR directly measures ranking quality, not just presence.
Key Takeaway: MRR gives you a single number (0 to 1) showing how consistently your RAG system surfaces relevant documents in top positions—higher is better, with 1.0 meaning perfect first-place results every time.