Course contentsShow
← AI Engineering
Lesson 357 of 1,8869. Retrieval-Augmented Generation (RAG) - BasicsFree lesson

Handling Empty Retrievals

Gracefully managing cases where retrieval returns no results or only low-quality matches.

Handling Empty Retrievals

What you'll learn: How to gracefully manage situations when your retrieval system finds nothing relevant to answer the user's question.

The Problem

Even with a well-designed RAG system, retrieval can fail. Your vector search might return zero results, or all returned chunks might fall below your relevance threshold. This happens when:

  • The user asks about information not in your knowledge base
  • The query is too vague or uses unfamiliar terminology
  • Your similarity threshold is set too high

Ignoring this scenario leads to confusing or hallucinated responses.

Why It Matters

When retrieval comes up empty, your LLM still needs to generate something. Without explicit instructions, the model might:

  • Fabricate an answer using its training data (hallucination)
  • Generate a generic, unhelpful response
  • Pretend it found relevant context when it didn't

The Solution

Be explicit in your prompt template. Include instructions for the empty-retrieval case:

Context: {context}

If the context above is empty or doesn't contain relevant information, 
politely inform the user that you don't have information about their 
question in your knowledge base. Do not make up answers.

Question: {question}

You can also implement upstream handling by detecting empty retrievals before calling the LLM and returning a pre-crafted message:

if not retrieved_docs or all(doc.score < threshold):
    return "I don't have information about that in my knowledge base."

This approach saves API costs and prevents hallucination.

Key Takeaway: Always handle empty retrievals explicitly—either through prompt instructions or pre-generation checks—to maintain honesty and avoid hallucinations when your knowledge base doesn't contain the answer.