Prompt-Based Filtering Instructions
What you'll learn: How to embed instructions directly in your prompts that tell the generation model to ignore irrelevant or low-quality retrieved context.
The Core Idea
You've already learned about LLM-based relevance filtering (where a separate LLM call judges context quality) and confidence thresholding (where you filter by retrieval scores). Prompt-based filtering takes a different approach: you instruct the generation model itself to identify and disregard irrelevant context within the same prompt where you're asking it to answer.
Instead of removing irrelevant chunks before they reach the model, you explicitly tell the model: "Here's some retrieved context. Some of it might not help answer the question. If a piece doesn't relate, ignore it and don't use it in your response."
Why This Works
Think of it like giving a research assistant a stack of documents with a note: "Not all of these are relevant—use your judgment." The model can apply contextual understanding to determine what's actually useful for the specific question, rather than treating all retrieved content as equally valid.
This approach is particularly useful when:
- You want to avoid multiple LLM calls for filtering
- Retrieved context quality varies significantly
- You trust the model to make relevance judgments during generation
Example Instructions
You might add to your RAG prompt template:
"If any of the provided context passages are not relevant to answering the user's question, simply ignore them. Only base your answer on information that directly addresses the question."
Key Takeaway: Prompt-based filtering instructions empower the generation model to selectively use retrieved context by explicitly authorizing it to disregard irrelevant material, reducing hallucination risk without requiring separate filtering steps.