What are chunking strategies in RAG, and how do you choose chunk size?

The short answer

Chunking splits documents into retrievable units; strategies include fixed-size windows, overlapping windows, and semantic or structure-aware splitting on sentences or sections. Smaller chunks improve retrieval precision but risk losing context, while larger chunks preserve context but dilute relevance, so chunk size and overlap are tuned to the content and the embedding model's context length.

How to think about it

Chunking splits documents into retrievable units; strategies include fixed-size windows, overlapping windows, and semantic or structure-aware splitting on sentences or sections. Smaller chunks improve retrieval precision but risk losing context, while larger chunks preserve context but dilute relevance, so chunk size and overlap are tuned to the content and the embedding model’s context length.

Learn it properly Advanced RAG

Keep practising

What chunking strategies exist for RAG and how do you choose between them? What is Retrieval-Augmented Generation (RAG) and how does a basic RAG pipeline work? When should you use RAG vs fine-tuning vs a long-context model? Compare RAG and fine-tuning. When would you use each? Your RAG system is hallucinating even though the correct context was retrieved. How do you debug it?

All NLP & LLMs questions

Explore further

RAG basics RAG with LangChain RAG evaluations

RAG Partitioning Vector Database Compaction