Course contentsShow
← Machine Learning and Deep Learning
Lesson 2027 of 3,53843. Retrieval-Augmented Generation: Advanced TechniquesFree lesson

Mean Reciprocal Rank (MRR)

Computing MRR to measure how quickly the first relevant document appears in retrieval results.

Mean Reciprocal Rank (MRR)

You'll learn how to measure retrieval quality by evaluating how quickly your system surfaces the first relevant document.

What Is MRR?

Mean Reciprocal Rank answers a simple question: On average, what position is the first correct answer? Unlike MAP or NDCG that consider all relevant documents, MRR focuses solely on the rank of the first relevant result. If your first relevant document appears at position 3, the reciprocal rank for that query is 1/3. Average this across all queries, and you get MRR.

Why MRR Matters in RAG

In many retrieval scenarios—especially question answering—users care most about finding one good answer fast. Think of web search: you're satisfied if the top result solves your problem. MRR captures this "time to first success" behavior, making it ideal when the first relevant document is what matters most.

The Formula

For a single query:

  • Reciprocal Rank (RR) = 1 / rank_of_first_relevant_doc

For multiple queries:

  • MRR = average of all reciprocal ranks

If no relevant document appears in your top-K results, that query contributes 0.

Concrete Example

Query 1: First relevant doc at position 2 → RR = 1/2 = 0.5
Query 2: First relevant doc at position 1 → RR = 1/1 = 1.0
Query 3: No relevant doc in top-K → RR = 0

MRR = (0.5 + 1.0 + 0) / 3 = 0.5

When to Use MRR

MRR works best when:

  • Only one relevant document exists per query
  • Users primarily engage with the first result
  • You're comparing retrieval systems on "speed to answer"

It's less useful when multiple relevant documents matter equally, where MAP or NDCG provide richer insights.

Key Takeaway: MRR measures how quickly your retrieval system surfaces the first relevant document—higher scores mean users find answers faster in your ranked results.