Hypothetical Document Embeddings (HyDE)
You'll learn how to improve retrieval by generating a fake answer first, then searching for documents similar to that answer.
The Core Idea
In traditional RAG, you embed your query ("What causes photosynthesis?") and search for documents with similar embeddings. But queries and documents often use different language — queries are questions, documents are statements.
HyDE flips this: instead of searching with your question, you ask the LLM to generate a hypothetical answer first (even if it hallucinates). Then you embed that fake answer and use it to retrieve real documents. Since both the hypothetical answer and actual documents are written in "answer language," they match better in embedding space.
How It Works
- User query arrives: "What are the health benefits of green tea?"
- Generate hypothetical document: Use an LLM to write a plausible answer (might be incorrect)
- "Green tea contains antioxidants called catechins that reduce inflammation..."
- Embed the hypothetical answer: Not the original query
- Retrieve similar documents: Find real documents close to this hypothetical answer's embedding
- Generate final answer: Use retrieved real documents to produce an accurate response
The Intuition
Think of it like asking a librarian for books about quantum physics. Instead of handing them your vague question, you first sketch what you think the answer looks like. The librarian then finds books that match your sketch's vocabulary and structure — even if your sketch was wrong, it's written in "physics textbook language" rather than "confused student language."
Key Takeaway: HyDE bridges the semantic gap between queries and documents by generating a hypothetical answer first, embedding it instead of the raw query, then retrieving documents similar to that answer's embedding — often improving retrieval quality despite the initial answer being fabricated.