Retrieval Phase: Query to Relevant Context
What You'll Learn
You'll learn how RAG systems transform user queries into embeddings and match them against stored document embeddings to surface the most relevant information.
The Search Problem
After documents have been chunked and indexed (as you learned previously), the retrieval phase handles incoming user queries. The goal: find which document chunks are most relevant to answer the user's question.
How Query Retrieval Works
When a user asks a question, the RAG system performs these steps:
- Embed the query: The same embedding model used during indexing converts the user's query into a vector representation
- Similarity search: This query vector is compared against all stored document chunk vectors
- Ranking: Chunks are scored by similarity (typically cosine similarity or dot product)
- Top-K selection: The system retrieves the K most similar chunks (e.g., top 3-5)
The Library Analogy
Think of it like a hyper-intelligent librarian. You ask "What causes ocean tides?" The librarian instantly translates your question into a conceptual fingerprint, then scans every book's fingerprint in the library to find the ones that match your conceptual need—not just keyword matches, but semantic meaning.
Why Embeddings Matter
Unlike keyword search, embeddings capture semantic similarity. The query "Why does the moon affect oceans?" will retrieve chunks about tidal forces even if those exact words don't appear—because the embedding space positions related concepts close together.
The retrieved chunks become the context fed to the LLM in the generation phase.
Key Takeaway: The retrieval phase converts user queries into embeddings and uses vector similarity to fetch the most semantically relevant document chunks from the indexed knowledge base.