Resolving Conflicting Retrieved Context
What You'll Learn
You'll learn how to programmatically handle contradictory information when your RAG system retrieves documents that disagree with each other.
The Problem: When Sources Clash
Imagine asking "What is the capital of Country X?" and retrieving:
- Document A (2015): "The capital is City1"
- Document B (2023): "After relocation, the capital is City2"
Your LLM receives both statements. Which should it trust? Without a strategy, the model might:
- Pick randomly
- Hedge awkwardly ("Some sources say...")
- Hallucinate a compromise
This is a common failure pattern when dealing with evolving facts, multiple perspectives, or low-quality sources.
Three Core Strategies
1. Source Credibility Weighting
Assign trust scores to documents based on origin. Academic papers, official documentation, or verified sources get higher weights. When conflicts arise, prioritize high-credibility sources.
Example: .gov domains > blog posts when discussing regulations.
2. Temporal Recency
Newer information typically supersedes older data for factual queries. Extract timestamps from metadata and prefer recent documents.
Example: A 2024 article about software features outweighs a 2020 tutorial.
3. Provenance Tracking
Maintain metadata about where and when information came from. Present conflicts transparently to users: "Source A says X (2020), but Source B says Y (2024)."
Implementation Approach
At retrieval time, attach metadata: {content, source, date, credibility_score}. Before generation, filter or reorder conflicting chunks based on these signals. You might instruct the LLM: "Prioritize the most recent information when documents conflict."
Key Takeaway: Conflicting context requires explicit resolution strategies—source credibility, temporal recency, and provenance tracking let you programmatically decide which information to trust rather than leaving it to chance.