Course contentsShow
← AI Engineering
Lesson 399 of 1,88610. RAG - Improving RetrievalFree lesson

Extractive Summarization for Compression

Selecting the most relevant sentences from retrieved chunks to reduce context size.

Extractive Summarization for Compression

You'll learn how to shrink retrieved context by pulling out only the most relevant sentences, keeping your RAG responses accurate while respecting token limits.

The Core Idea

After retrieving multiple chunks and potentially reranking them, you may still have too much text to fit comfortably in your LLM's context window. Extractive summarization solves this by selecting the most important sentences from your retrieved chunks rather than including everything.

Think of it like highlighting a textbook: instead of reading every word on the page, you scan for the sentences that directly answer your question and ignore the rest. You're not rewriting anything—just cherry-picking the gems.

How It Works

Unlike abstractive summarization (which rewrites content), extractive summarization copies existing sentences from the source. The process:

  1. Retrieve and rerank your top chunks
  2. Score each sentence based on relevance to the query
  3. Select the top N sentences that fit your token budget
  4. Pass only these sentences to the LLM as context

This preserves the original wording (reducing hallucination risk) while dramatically reducing token count.

When to Use It

Extractive compression shines when:

  • You have verbose documents but need concise context
  • Token costs are a concern
  • You've already done reranking but still have too much text
  • You want to preserve exact phrasing from sources (for citations)

The trade-off? You might lose valuable supporting context that connects those sentences. Balance compression aggressively enough to save tokens, but not so much that you strip away the answer.

Key Takeaway: Extractive summarization picks the most relevant sentences from retrieved chunks, giving you a token-efficient context that preserves original wording while reducing noise for your LLM.