Normalized Discounted Cumulative Gain (NDCG)
What you'll learn: How to evaluate retrieval systems when relevance isn't just binary but comes in grades, and why ranking matters as much as finding the right documents.
The Problem with Binary Metrics
Recall and Precision treat documents as either relevant or not. But in real retrieval scenarios, some documents are highly relevant, others are somewhat relevant, and some are marginally helpful. NDCG captures this nuance and penalizes systems that bury highly relevant documents deep in the results.
How NDCG Works
Think of NDCG like grading a restaurant recommendation list. If your friend asks for "amazing Italian restaurants" and you list one great place at position 10, that's worse than putting it at position 1—even though both lists contain it.
Discounted Cumulative Gain (DCG) sums relevance scores but applies a discount based on rank position:
DCG = rel₁ + rel₂/log₂(2) + rel₃/log₂(3) + ... + relₖ/log₂(k)
Where rel is the graded relevance score (e.g., 0=irrelevant, 1=somewhat relevant, 3=highly relevant).
Normalization compares your system's DCG to the ideal DCG (IDCG)—what you'd get if you perfectly sorted documents by relevance:
NDCG = DCG / IDCG
NDCG ranges from 0 to 1. A score of 1.0 means perfect ranking; lower scores indicate relevant documents are poorly positioned.
Why This Matters for RAG
In RAG systems, retrieving a highly relevant chunk at position 10 is less useful than at position 1—the LLM might not even see it if your context window is limited. NDCG directly measures whether your hybrid search + reranking pipeline surfaces the best documents first.
Key Takeaway: NDCG evaluates retrieval quality with graded relevance, penalizing systems that rank highly relevant documents lower—essential for optimizing RAG pipelines where position and relevance degree both matter.