Course contentsShow
← AI Engineering
Lesson 380 of 1,88610. RAG - Improving RetrievalFree lesson

Evaluating Query Optimization Impact

Measuring retrieval quality improvements from query optimization using recall, precision, and end-to-end metrics.

Evaluating Query Optimization Impact

You'll learn: How to measure whether your query optimization techniques (like HyDE, query expansion, or multi-query) actually improve retrieval quality.

Why Measurement Matters

You've learned many query optimization tricks—from synonym expansion to step-back prompting. But how do you know if they're working? Without measurement, you're flying blind. Evaluation tells you whether your optimizations help, hurt, or waste compute resources.

Core Metrics to Track

Recall: Of all the relevant documents that exist in your knowledge base, what percentage did you retrieve? Higher is better. Query expansion might boost recall by catching synonym matches.

Precision: Of the documents you retrieved, what percentage are actually relevant? If HyDE generates a hypothetical document that's too specific, precision might drop because you pull in near-misses.

End-to-End Quality: Retrieval metrics only tell half the story. Does the final LLM answer improve? Track:

  • Answer correctness (human eval or LLM-as-judge)
  • Citation accuracy
  • User satisfaction signals

The Comparison Framework

Always compare against a baseline:

  1. Run your RAG system with plain queries (no optimization)
  2. Run it again with optimized queries (multi-query, HyDE, etc.)
  3. Measure recall/precision on a labeled test set
  4. Compare final answer quality

Example: If query expansion increases recall from 60% to 75% but precision drops from 90% to 70%, you might be retrieving more noise. Check if the end-to-end answer quality actually improved—sometimes more context helps the LLM despite lower precision.

The Trade-off Reality

Every optimization has costs: latency, API calls, complexity. If HyDE adds 500ms but only improves answer quality by 2%, maybe it's not worth it for real-time chat.

Key Takeaway: Use recall and precision to measure retrieval improvements, but always validate end-to-end answer quality—metrics mean nothing if users get worse results.