Course contentsShow
← AI Engineering
Lesson 347 of 1,8869. Retrieval-Augmented Generation (RAG) - BasicsFree lesson

Evaluating Chunking Strategies

Testing different chunking approaches against retrieval quality and downstream generation performance.

Evaluating Chunking Strategies

What you'll learn: How to systematically test different chunking approaches to find what actually works best for your RAG system.

The Testing Challenge

You've learned multiple chunking strategies—fixed-size, recursive, semantic, sentence-based—but which one should you actually use? The answer isn't theoretical; it requires measurement. Evaluating chunking strategies means running experiments that test how well each approach supports both retrieval quality (finding the right chunks) and downstream generation performance (producing good answers).

The Two-Level Evaluation

Think of chunking evaluation like testing cooking techniques: does your method keep ingredients intact (retrieval), and does the final dish taste good (generation)?

Retrieval Quality Metrics:

  • How often do retrieved chunks contain the information needed to answer the query?
  • Are relevant chunks ranked highly enough (within your top-K)?
  • How precise are results—minimal irrelevant chunks?

Generation Performance Metrics:

  • Does the LLM produce correct, complete answers?
  • Are responses coherent and well-grounded in the retrieved context?
  • How often does the system hallucinate or miss key details?

Running Comparisons

Create a test set of representative queries with known correct answers. Then systematically compare strategies: fixed 512-token chunks vs. semantic chunking vs. sentence-based with overlap. Measure each on both levels. You might discover that semantic chunking retrieves better chunks but creates fragments too small for the LLM, or that overlap helps retrieval but adds redundancy that confuses generation.

The "best" strategy depends on your specific documents, queries, and quality requirements.

Key Takeaway: Evaluating chunking strategies requires testing both retrieval accuracy and final generation quality against real queries—never assume one approach fits all use cases.