The RAG Pipeline: Three Core Stages
What you'll learn: How Retrieval-Augmented Generation breaks down into three distinct stages that transform external data into enriched LLM responses.
The Three-Stage Data Flow
Now that you understand what RAG solves—knowledge cutoff and hallucination—let's see how it actually works. RAG is not a single black box; it's a pipeline with three clear stages that data flows through sequentially.
1. Indexing Stage
Before you can retrieve anything, you need to prepare your knowledge base. This stage happens once, upfront:
- Break documents into chunks (paragraphs, sections)
- Convert text chunks into vector embeddings
- Store embeddings in a vector database with metadata
Think of this as building a library's card catalog system—you organize books once so you can find them quickly later.
2. Retrieval Stage
When a user asks a question, this stage activates in real-time:
- Convert the user's query into an embedding
- Search the vector database for most similar chunks
- Return top-k relevant passages (typically 3-10)
This is like a librarian finding the most relevant books for your research question.
3. Generation Stage
Finally, the LLM produces the actual response:
- Inject retrieved passages into the prompt context
- The LLM reads both the query and retrieved knowledge
- Generate an answer grounded in the provided evidence
The LLM acts like a student who can now cite sources instead of guessing from memory.
Why Three Stages?
Separating these stages allows you to optimize each independently. You can swap vector databases (indexing), tune retrieval algorithms (retrieval), or change LLM models (generation) without rebuilding everything.
Key Takeaway: RAG pipelines flow through indexing (prepare knowledge once), retrieval (find relevant context per query), and generation (produce grounded answers)—each stage solving a distinct problem in the knowledge-augmentation process.