---
title: "What is chunking?"
excerpt: "Splitting documents into passages before embedding them. The unglamorous step that quietly decides whether your RAG system can retrieve a coherent fact at all."
definition: "Chunking is the process of splitting source documents into smaller passages before embedding them, so retrieval can return focused, self-contained units of text rather than whole documents."
date: "2026-07-09"
lastModified: "2026-07-09"
author: "Teo Deleanu"
authorAvatar: "/team/teo.jpg"
tags: ["RAG", "AI Engineering", "Retrieval", "Chunking"]
keywords:
  - "what is chunking"
  - "chunking for rag"
  - "chunk size embeddings"
---

Chunking is how you decide the unit of retrieval. Before you [embed](/glossary/embeddings) a document, you split it into passages — chunks — and each chunk becomes one searchable vector. Get the split right and retrieval returns a coherent, self-contained fact. Get it wrong and it returns half a sentence or an unfocused blur.

## Why it matters in production

Chunk size is a tradeoff you cannot avoid. Chunks too large produce vague [embeddings](/glossary/embeddings) that match many queries weakly and match none precisely. Chunks too small sever the context a fact needs — the answer is in chunk 4 but the qualifying condition was in chunk 3, and they never travel together. The chunk is the atom your whole [RAG](/glossary/retrieval-augmented-generation) system is built from, and a bad atom poisons everything downstream.

## The common mistake

Splitting on a fixed character count with no regard for structure. Cutting mid-sentence or mid-table destroys meaning, and a fact split across a naive boundary becomes unretrievable — no query matches either fragment well. The failure shows up as "the model doesn't know something that is clearly in the docs," and the cause is a boundary, not the model.

## How we actually use it

We chunk along the document's natural structure — sections, headings, logical units — rather than blind character counts, and we add overlap so a fact near a boundary survives in both neighbors. Then we validate it the only way that counts: does a question we expect to answer actually retrieve the chunk that answers it? That check, part of [how RAG works in production](/blog/how-rag-works-in-production), turns chunking from guesswork into something measurable.