Contextual Query Expansion from Chat History
What you'll learn: How to use previous conversation turns to resolve ambiguous references in follow-up queries, making retrieval more accurate in multi-turn RAG dialogues.
The Challenge of Conversational Continuity
When users engage in multi-turn conversations with RAG systems, they naturally use pronouns ("it," "they"), implicit references ("the second one"), and contextual shortcuts ("what about now?"). These queries are perfectly clear to humans following the conversation but utterly confusing to a retrieval system that only sees the current question.
How Contextual Query Expansion Works
Instead of sending the raw follow-up query to your retrieval system, you first expand it by incorporating relevant context from previous turns. This transforms an underspecified query into a standalone, self-contained version.
Example conversation:
- Turn 1: "What are the benefits of solar panels?"
- Turn 2: "How much do they cost?"
Without expansion, turn 2 retrieves documents about generic costs. With expansion, you rewrite it to: "How much do solar panels cost?" — now retrieval works correctly.
Implementation Approach
You can use an LLM to rewrite queries by providing:
- The chat history (last 2-3 turns typically)
- The current query
- A prompt asking for a standalone reformulation
The LLM resolves pronouns, incorporates mentioned entities, and makes implicit topics explicit. This expanded query then goes to your vector database or hybrid search system.
Balancing Context Window
Don't include the entire conversation — just enough recent context to resolve ambiguity. Too much history adds noise and latency; too little misses critical references.
Key Takeaway: Contextual query expansion transforms conversational queries into standalone versions by using chat history to resolve pronouns and implicit references, dramatically improving retrieval accuracy in multi-turn RAG systems.