Course contentsShow
← AI Engineering
Lesson 797 of 1,88620. Evaluation Frameworks for LLM SystemsFree lesson

Retrieval Quality Metrics

Evaluating RAG systems with precision@k, recall@k, MRR, and NDCG to measure search and ranking effectiveness.

Retrieval Quality Metrics

What you'll learn: How to measure whether your RAG system finds and ranks the right documents using industry-standard metrics.

Why Measure Retrieval Quality?

When building RAG (Retrieval-Augmented Generation) systems, you need to know if your search component actually finds relevant information. Just like grading a search engine, you want to measure both what it retrieves and how well it ranks results.

Core Metrics Explained

Precision@k

Precision@k measures: "Of the top k results returned, how many were actually relevant?"

If you retrieve 5 documents and 3 are relevant, precision@5 = 3/5 = 0.6. It answers: "Can I trust what's at the top?"

Recall@k

Recall@k measures: "Of all relevant documents that exist, how many did the top k results capture?"

If 10 relevant docs exist and your top 5 captures 3 of them, recall@5 = 3/10 = 0.3. It answers: "Am I missing important information?"

MRR (Mean Reciprocal Rank)

MRR measures how quickly you hit the first relevant result. If the first relevant doc appears at position 3, the reciprocal rank = 1/3. Average this across queries to get MRR. Higher is better—users want answers fast.

NDCG (Normalized Discounted Cumulative Gain)

NDCG is sophisticated: it considers how relevant each result is (not just yes/no) and where it appears (position matters). Documents at position 1 count more than position 10. NDCG ranges 0-1, where 1.0 means perfect ranking.

Choosing the Right Metric

  • Use precision@k when accuracy matters most (medical, legal)
  • Use recall@k when completeness matters (research, investigation)
  • Use MRR for single-answer questions (FAQ bots)
  • Use NDCG when relevance has degrees and ranking quality matters

Key Takeaway: Retrieval quality metrics like precision@k, recall@k, MRR, and NDCG help you objectively measure whether your RAG system finds the right documents in the right order.