Course contentsShow
← AI Engineering
Lesson 355 of 1,8869. Retrieval-Augmented Generation (RAG) - BasicsFree lesson

Context Relevance Instructions

Prompting the LLM to prioritize relevant retrieved documents and ignore irrelevant information.

Context Relevance Instructions

What you'll learn: How to instruct the LLM to focus on relevant retrieved documents and discard noise in RAG systems.

The Problem: Not All Retrieved Chunks Are Equally Useful

Even with careful chunking and retrieval, your top-K search results often contain some irrelevant or tangential information. Maybe the vector similarity was high due to keyword overlap, but the actual content doesn't answer the user's question.

If you simply dump all retrieved documents into the LLM's context window, it might:

  • Get distracted by irrelevant details
  • Generate answers based on weak evidence
  • Waste tokens on processing noise

The Solution: Explicit Relevance Filtering Instructions

Context relevance instructions are prompt directives that tell the LLM to actively filter and prioritize the context you've provided.

Think of it like giving a researcher a stack of papers with a note: "Some of these articles might be off-topic. Only use the ones that directly address the question. If none are relevant, say so."

Example Prompt Pattern

You are given several document excerpts below. 
Some may be relevant to the user's question, others may not be.

**Instructions:**
- Only use information from excerpts that DIRECTLY answer the question
- Ignore excerpts that are tangential or off-topic
- If no excerpts contain relevant information, say "I don't have enough information to answer that"

**Question:** {user_query}

**Retrieved Documents:**
{retrieved_chunks}

This approach combines with your earlier work on prompt template design and limiting retrieved context to create a robust filtering layer at inference time.

Key Takeaway: Explicit context relevance instructions guide the LLM to self-filter retrieved documents, reducing hallucinations and improving answer quality when some retrieved chunks are noisy or irrelevant.