Course contentsShow
← Machine Learning and Deep Learning
Lesson 1954 of 3,53842. Retrieval-Augmented Generation: FoundationsFree lesson

Naive RAG Architecture and Its Limitations

Implementing a basic RAG pipeline and understanding common failure modes like irrelevant retrieval and context overflow.

Naive RAG Architecture and Its Limitations

What you'll learn: You'll understand how to implement a basic RAG pipeline and recognize common failure modes that plague simple retrieval systems.

The Naive RAG Pipeline

A "naive" RAG system implements the three stages you've learned—indexing, retrieval, generation—in their simplest form:

  1. Chunk documents into fixed-size pieces (e.g., 512 tokens)
  2. Embed chunks using a bi-encoder
  3. Retrieve top-K most similar chunks for any query
  4. Stuff all retrieved context into the LLM prompt
  5. Generate an answer

This straightforward approach works surprisingly well for simple cases, but quickly reveals critical weaknesses.

Common Failure Modes

Irrelevant Retrieval

Your retrieval system might return chunks that match keywords but miss semantic intent. For example, querying "How does photosynthesis work?" might retrieve chunks mentioning "photosynthesis" but discussing photosynthetic efficiency in solar panels rather than biological processes. The LLM then generates answers from misleading context.

Context Overflow

Retrieving too many chunks exhausts the LLM's context window, forcing truncation that may cut critical information. Conversely, retrieving too few risks missing relevant details spread across multiple chunks.

Chunk Boundary Problems

Fixed-size chunking often splits coherent explanations mid-sentence or separates related paragraphs, providing incomplete context to the LLM.

No Reranking or Filtering

Naive RAG trusts similarity scores blindly—it doesn't verify whether retrieved chunks actually answer the query or contain contradictory information.

Key Takeaway: Naive RAG provides a functional baseline but suffers from irrelevant retrieval, context overflow, poor chunking, and lack of quality filtering—limitations that advanced RAG techniques specifically address.