Multi-Hop Reasoning Failures
What you'll learn: How to identify when a single retrieval pass fails because your question requires connecting information scattered across multiple documents.
The Single-Hop Limitation
Imagine asking "How did the founder's previous company experience influence their current product strategy?" A standard RAG system retrieves the top-K most relevant chunks and passes them to the LLM. But what if the founder's background lives in one document, and the product strategy in another? Single-pass retrieval often returns either context about the founder or strategy details — rarely both in the top results.
This is a multi-hop reasoning failure: the question requires synthesizing facts from separate sources that don't individually match the query well.
Why This Happens
Vector similarity searches for semantic closeness to your query. But multi-hop questions like "What's the revenue impact of the policy change announced after the CEO transition?" need:
- Facts about the CEO transition (Document A)
- The policy change timeline (Document B)
- Revenue data from that period (Document C)
No single chunk scores highly for all three aspects simultaneously. You retrieve fragments of the puzzle, but miss critical connecting pieces.
Diagnosing Multi-Hop Failures
Look for these warning signs:
- The LLM says "I don't have enough information" despite relevant docs existing
- Retrieved chunks address part of the question but lack linking facts
- Questions with "because," "impact of," "relationship between" consistently fail
- Manual inspection shows needed info exists but wasn't co-located in any chunk
Key Takeaway: Multi-hop reasoning failures occur when complex queries require synthesizing information from multiple documents that don't naturally appear together in top-K retrieval results — a sign you need iterative or decomposed retrieval strategies.