Paragraph-Based Chunking
What you'll learn: How to split documents at paragraph boundaries to maintain natural topic flow and preserve document structure in RAG systems.
When you previously learned sentence-based chunking, you split text at sentence boundaries. But sentences are sometimes too granular—a single concept often spans multiple sentences. Paragraph-based chunking uses paragraph breaks as natural split points, treating each paragraph (or small groups of paragraphs) as a chunk.
Why Paragraph Boundaries?
Writers structure documents hierarchically. A paragraph typically contains a single coherent idea or sub-topic. Breaking at paragraph boundaries:
- Preserves topical coherence: Related sentences stay together
- Respects document structure: Headers, sections, and logical divisions remain intact
- Reduces fragmentation: Unlike fixed-size chunks that might split mid-paragraph
Think of it like cutting a sandwich along its natural layers (bread, filling, lettuce) rather than making arbitrary horizontal slices.
How It Works
Identify paragraph delimiters (usually double newlines: \n\n or \r\n\r\n in plain text, <p> tags in HTML, or style markers in PDFs). Each paragraph becomes a candidate chunk. If a paragraph is too large, you might split it further; if too small, combine consecutive paragraphs until reaching a reasonable token count.
Trade-offs
- Variable chunk sizes: Paragraphs vary in length, so some chunks may be too short (lacking context) or too long (exceeding LLM context limits)
- Better semantic integrity: Each chunk is more likely to be self-contained and meaningful
- Document-dependent: Works best with well-structured documents; informal text (chat logs, social media) may lack clear paragraph boundaries
Key Takeaway: Paragraph-based chunking leverages the document's natural structure to create semantically coherent chunks, balancing retrieval precision with contextual completeness—ideal for well-formatted documents like articles, reports, and books.