Course contentsShow
← Machine Learning and Deep Learning
Lesson 1989 of 3,53842. Retrieval-Augmented Generation: FoundationsFree lesson

Semantic Chunking

Using embeddings to identify semantic boundaries and group semantically similar sentences into coherent chunks.

Semantic Chunking

What you'll learn: How to use embeddings to create chunks that respect the natural semantic boundaries in your text, grouping related sentences together.

The Core Idea

Up until now, you've seen chunking strategies based on mechanical rules: fixed character counts, sentence breaks, or paragraph boundaries. Semantic chunking takes a smarter approach—it uses embeddings to measure the meaning of sentences and groups them based on semantic similarity.

Think of it like organizing a messy desk. Fixed-size chunking is like dividing items into equal piles regardless of what they are. Semantic chunking is like grouping pens together, papers together, and cables together—respecting what naturally belongs with what.

How It Works

  1. Embed each sentence individually using your embedding model
  2. Calculate similarity between consecutive sentences (cosine similarity between their embeddings)
  3. Identify semantic boundaries where similarity drops significantly—these mark topic shifts
  4. Group sentences between boundaries into coherent chunks

For example, if sentence 5 and 6 have low similarity (0.4) but sentences 6-9 are all highly similar (>0.8), you'd split before sentence 6 and keep 6-9 together.

Why This Matters for RAG

When your retrieval system fetches a chunk, you want that chunk to contain a complete, coherent idea—not half of one topic and half of another. Semantic chunking ensures each chunk is semantically self-contained, improving the quality of context passed to your LLM.

The trade-off? It's computationally more expensive than simple sentence-based chunking since you must embed every sentence and compute pairwise similarities.

Key Takeaway: Semantic chunking uses embedding similarity to detect natural topic boundaries, creating chunks that preserve semantic coherence rather than arbitrary text divisions.