Course contentsShow
← AI Engineering
Lesson 443 of 1,88611. RAG - Improving GenerationFree lesson

Common RAG Failure Modes

Identify the most frequent failure patterns: no relevant docs, wrong docs retrieved, hallucination despite context, and citation errors.

Common RAG Failure Modes

You'll learn: The most frequent failure patterns in RAG systems and how to recognize them before they reach users.

Understanding What Goes Wrong

Even well-designed RAG systems fail in predictable ways. Identifying these patterns is the first step toward building robust error handling and debugging workflows. Think of these as the "usual suspects" when your RAG system produces poor answers—knowing them helps you diagnose problems faster.

The Four Core Failure Modes

1. No Relevant Documents Retrieved

The retrieval step returns empty results or documents completely unrelated to the query. Your vector search finds nothing useful, often due to vocabulary mismatch, poor embeddings, or missing data.

2. Wrong Documents Retrieved

The retrieval returns something, but it's incorrect or only tangentially related. The model then generates answers from irrelevant context, leading to confusing or misleading responses.

3. Hallucination Despite Context

The model ignores perfectly good retrieved context and invents information anyway. This happens when the model "trusts" its training data over the provided documents, or when context placement is suboptimal.

4. Citation Errors

The system provides a correct answer but cites the wrong source, fails to cite at all, or invents non-existent references. This breaks trust even when the content is accurate.

Why This Matters

Each failure mode requires different solutions: better retrieval for #1 and #2, prompt engineering for #3, and citation instruction improvements for #4. Misdiagnosing which failure you're facing wastes debugging time.

Key Takeaway: The four main RAG failures are no relevant docs, wrong docs retrieved, hallucination despite good context, and citation errors—each needs targeted fixes, so accurate diagnosis is critical.