Corrective RAG (CRAG): Evaluating Retrieved Context
What you'll learn: How to use an LLM as a judge to evaluate retrieved context quality and automatically trigger fallback strategies like web search when the retrieved information falls short.
The Core Problem
Even after retrieving and reranking documents, your RAG system might still serve up irrelevant, outdated, or insufficient context. Instead of blindly passing everything to the generation step, Corrective RAG (CRAG) adds a self-correction layer that asks: "Is this retrieved context actually good enough to answer the question?"
How CRAG Works
Think of CRAG as a quality inspector on an assembly line. Before context reaches the final generation stage, an LLM evaluates each retrieved chunk and assigns one of three verdicts:
- Correct – Context is highly relevant; proceed with generation
- Ambiguous – Context has some relevance; use it but compress or refine it first
- Incorrect – Context is irrelevant or insufficient; trigger alternative retrieval (like web search)
This evaluation step uses a simple prompt asking the LLM to judge relevance against the original query. Based on the verdict, your system can:
- Strip out irrelevant chunks before generation
- Trigger web search when internal knowledge is lacking
- Route to specialized indexes for better results
- Combine internal and external sources intelligently
The Analogy
Imagine asking a librarian for books on quantum computing, and they hand you cookbooks. A traditional RAG system would try to answer your question using those cookbooks. CRAG is like having a supervisor who checks the books first and says, "These aren't right—let's search the university library instead."
Why This Matters
CRAG reduces hallucinations by catching bad retrieval early. It's particularly valuable for questions requiring current information, niche domains, or when your document collection has coverage gaps.
Key Takeaway: Corrective RAG uses an LLM as a quality gate to evaluate retrieved context and automatically trigger fallback strategies like web search when internal retrieval fails, preventing bad context from reaching generation.