Course contentsShow
← Machine Learning and Deep Learning
Lesson 2053 of 3,53843. Retrieval-Augmented Generation: Advanced TechniquesFree lesson

Adaptive Chunk Selection

Dynamically deciding how many and which retrieved chunks to include in the context window.

Adaptive Chunk Selection

What you'll learn: How RAG systems can intelligently decide which and how many retrieved chunks to pass to the LLM based on relevance, quality, and context constraints.

The Core Problem

Not all retrieved chunks deserve equal attention. After retrieving candidate passages from your vector database and potentially reranking them, you face a critical decision: which chunks should actually enter the LLM's context window? Simply taking "top-k" chunks wastes tokens on marginally relevant content and risks context overflow.

Adaptive chunk selection dynamically adjusts both the number and identity of chunks based on real-time quality assessment.

How It Works

Think of it like a smart bouncer at an exclusive club—not everyone who shows up gets in, even if they were invited. The system evaluates each retrieved chunk against criteria like:

  • Relevance threshold: Only chunks scoring above a dynamic cutoff make it through
  • Diminishing returns: Stop adding chunks when marginal relevance drops sharply
  • Semantic diversity: Skip redundant chunks that repeat information
  • Token budget awareness: Adjust selection based on remaining context window space
  • Query complexity signals: Simple questions might need 2-3 chunks; complex multi-hop queries might justify 10+

For example, if your top-ranked chunk scores 0.92 but the 6th chunk scores 0.45, adaptive selection might include only the top 3-4 rather than blindly passing all 10 retrieved candidates.

Why It Matters

This addresses context window overflow, improves answer quality by reducing noise, and optimizes latency by processing fewer tokens. It bridges your earlier work on reranking with the practical constraint that LLMs have finite attention spans.

Key Takeaway: Adaptive chunk selection transforms retrieval from a rigid "top-k" operation into an intelligent filtering process that respects both relevance quality and resource constraints.