Retrieval Evaluation Fundamentals
What you'll learn: Why measuring retrieval quality is critical for RAG systems and which metrics tell you if your retrieval is actually working.
Why Retrieval Quality Matters
In RAG systems, your LLM can only be as good as the context it receives. If your retrieval stage pulls irrelevant documents, the best prompt engineering won't save your generation quality. Think of retrieval as a librarian finding books for your LLM to read—if the librarian brings the wrong books, even the smartest reader will give poor answers.
Poor retrieval causes hallucinations, incomplete answers, and wasted tokens on irrelevant context. That's why systematic evaluation is essential.
Core Retrieval Metrics
Recall@K measures: Of all the relevant documents in your database, what percentage did you retrieve in your top-K results? If 10 documents could answer the query and you retrieved 7 of them in your top-10, that's 70% recall. High recall means you're not missing important information.
Precision@K measures: Of the K documents you retrieved, what percentage are actually relevant? If you retrieved 10 documents but only 6 matter, that's 60% precision. High precision means you're not polluting the LLM's context with noise.
Mean Reciprocal Rank (MRR) measures: How quickly do relevant results appear? If the first relevant document is at position 3, the reciprocal rank is 1/3. Averaged across queries, this tells you if your best results surface early—critical when you only pass top-3 to your LLM.
Normalized Discounted Cumulative Gain (NDCG) goes further by accounting for how relevant each document is and penalizing relevant docs that appear late in rankings.
Key Takeaway: Retrieval evaluation metrics like Recall@K, Precision@K, MRR, and NDCG quantify whether your RAG system finds the right documents in the right order—the foundation of reliable, factual generation.