Course contentsShow
← Machine Learning and Deep Learning
Lesson 2037 of 3,53843. Retrieval-Augmented Generation: Advanced TechniquesFree lesson

Handling Low-Quality Retrieval Results

Detect and filter irrelevant chunks using confidence thresholds, semantic similarity cutoffs, and cross-encoder reranking.

Handling Low-Quality Retrieval Results

What you'll learn: How to detect and filter out irrelevant chunks before they pollute your LLM's context window.

The Problem with Noisy Retrieval

Even well-tuned retrieval systems sometimes return chunks that seem relevant by embedding similarity but actually contain unhelpful or off-topic information. Think of it like searching for "apple pie recipe" and getting results about Apple Inc.'s quarterly financials—technically related to "apple," but useless for your actual question.

Three Defense Mechanisms

1. Confidence Thresholds

Set a minimum similarity score below which chunks are rejected. If your top result only scores 0.62 when typical good matches score 0.85+, that's a red flag. You're essentially saying: "If the retrieval system isn't confident, don't use it."

2. Semantic Similarity Cutoffs

Instead of absolute thresholds, compare the score gap between your top results. If the best chunk scores 0.80 but the second-best scores 0.79, they're probably equally relevant. But if it's 0.80 vs 0.55, only trust the first one.

3. Cross-Encoder Reranking

This is your quality gate. After initial retrieval, run a cross-encoder that jointly processes the query and each candidate chunk. Cross-encoders are slower but more accurate—they can detect subtle irrelevance that bi-encoders miss. Filter out anything below your cross-encoder's threshold before sending context to the LLM.

The Safety Net Pattern

Combine all three: retrieve with bi-encoders → filter by threshold → rerank with cross-encoder → apply final cutoff. This layered defense prevents "garbage in, garbage out" at generation time. Better to tell the user "I couldn't find relevant information" than to hallucinate based on poor context.

Key Takeaway: Low-quality retrieval results sabotage RAG output—use confidence thresholds, similarity gaps, and cross-encoder reranking as progressive filters to ensure only genuinely relevant context reaches your LLM.