Precision and Recall for Retrieval
What You'll Learn
You'll learn how to measure retrieval quality by calculating what fraction of retrieved documents are relevant (precision) and what fraction of all relevant documents you found (recall).
The Measurement Challenge
When your RAG system retrieves documents, you need to answer two critical questions:
- Of what I retrieved, how much was actually useful? (Precision)
- Of all the useful documents available, how many did I find? (Recall)
Think of it like fishing: precision tells you what percentage of your catch is the fish you wanted (vs. junk), while recall tells you what percentage of all the desired fish in the pond you actually caught.
Computing Precision and Recall
Precision = (Relevant documents retrieved) / (Total documents retrieved)
If you retrieve 10 documents and 7 are relevant, your precision is 7/10 = 0.70 or 70%.
Recall = (Relevant documents retrieved) / (Total relevant documents that exist)
If there are 20 relevant documents in your corpus and you retrieved 7 of them, your recall is 7/20 = 0.35 or 35%.
The Trade-off
These metrics often conflict. Retrieving more documents (higher top-K) typically increases recall (you catch more relevant ones) but decreases precision (you also catch more irrelevant ones). Retrieving fewer documents does the opposite.
For RAG systems, high precision keeps your LLM context clean and focused, while high recall ensures you don't miss critical information. Your choice depends on your use case: medical diagnosis might prioritize recall (don't miss anything), while quick factual answers might prioritize precision (only show confident results).
Key Takeaway: Precision measures how accurate your retrieved results are, while recall measures how complete they are—use both to understand retrieval quality trade-offs in your RAG pipeline.