Chunking: Definition and Examples

Chunking, or segmentation, is a fundamental strategy in prompt engineering and AI information processing. It consists of dividing large content or a complex task into smaller, more digestible, and easier-to-process pieces ("chunks"). This approach is directly inspired by cognitive science, where chunking refers to the human brain's ability to group information into meaningful units to better memorize and manipulate it.

In the context of large language models (LLMs), chunking applies at two distinct levels. First, at the data level: when feeding a RAG (Retrieval-Augmented Generation) system, source documents are split into optimally sized segments to be indexed and retrieved efficiently. The size and overlap of these segments directly influence the relevance of results. Second, at the prompt level: facing a complex task, one decomposes it into sequential subtasks that the model processes one by one.

Chunking at the prompting level helps circumvent several LLM limitations: the limited context window size, the tendency to lose precision on long tasks, and the difficulty maintaining coherence over multiple instructions. By processing each segment independently or sequentially, one obtains more accurate and better-structured responses.

The quality of chunking depends on several factors: the chosen granularity (too fine, context is lost; too broad, precision is lost), the semantic coherence of each segment, and the overlap strategy between adjacent segments. Mastering chunking is essential for anyone working with LLMs on non-trivial tasks.