Chunking strategies for RAG, and why we rewrite chunks into facts

October 9, 2026

A RAG pipeline cuts documents into chunks, embeds them, and hands the best matches to a model. Most of the advice about chunking is about where to cut. Bacopa cuts too, but then a model rewrites each piece into a fact that an agent can read without the page around it. This post compares the common strategies, shows what they leave unsolved, and says what our approach costs.

The common chunking strategies

• Fixed-size: windows of N tokens or characters, often with some overlap so a sentence cut at one edge appears whole in the next window.
• Recursive: split on paragraphs, then lines, then sentences, then words, until each piece fits the size limit. It keeps paragraphs whole when it can.
• Semantic: embed each sentence and cut where the meaning of neighboring sentences drifts apart.
• Hierarchical, or structure-based: cut at the document's own headings, and keep the heading path with each chunk.
• Late chunking: embed the whole document with a long-context model first, then pool the token embeddings of each chunk, so a chunk's vector carries some of the document's context.
• Agentic chunking: a model reads the text and decides where chunks end, often by grouping short statements drawn from it.

Each step down the list cuts in a better place. None of them changes the text inside the chunk, and that text was written for a reader who has the rest of the page.

What a chunk looks like

Take Cloudflare's documentation page on batching, retries and delays in Cloudflare Queues. Here is one 1,000-character window of it, as a fixed-size chunker would cut it:

calls to `ack()`, `retry()` and their `ackAll()` / `retryAll()` equivalents follow the below precedence rules:

* If you call `ack()` on a message, subsequent calls to `ack()` or `retry()` are silently ignored.
* If you call `retry()` on a message and then call `ack()`: the `ack()` is ignored. The first method call wins in all cases.
* If you call either `ack()` or `retry()` on a single message, and then either/any of `ackAll()` or `retryAll()` on the batch, the call on the single message takes precedence. That is, the batch-level call does not apply to that message (or messages, if multiple calls were made).

## Delivery failure

When a message is failed to be delivered, the default behaviour is to retry delivery three times before marking the delivery as failed. You can set `max_retries` (defaults to 3) when configuring your consumer, but in most cases we recommend leaving this as the default.

Messages that reach the configured maximum retries will be deleted from the queue, or if

The chunk starts in the middle of a sentence, holds the end of one section and the start of the next, and stops on "or if". Overlap and better boundaries fix those three. The fourth problem stays: the chunk never names Cloudflare Queues. A search for "Cloudflare Queues retry limit" has to match it on "retry" alone, and an agent that gets it doesn't know which product's messages it is reading about.

Pages carry noise too. The first window of this same page holds "Skip to content", a note about the docs index, and a row of "Copy as Markdown" and "View as Markdown" links. That text gets embedded, ranked and read like everything else.

Contextual retrieval adds context to the chunk

Anthropic's contextual retrieval attacks the fourth problem. Before indexing, a model writes 50 to 100 tokens that place each chunk in its document, and that text is prepended to the chunk for both embedding and BM25. Anthropic reports 49% fewer failed retrievals, and 67% with reranking. It keeps the original chunk, so the agent still reads the half sentences and the noise, now with a summary in front of them.

Bacopa rewrites each section into a fact

Bacopa keeps the structure-based cut and replaces what goes inside. It cuts the page's text at its h1 and h2 headings. A section under 200 tokens joins the next one, and a section over 3,000 tokens is cut at its h3 headings or at sentence boundaries. Then a model reads the sections and writes one fact per section.

Here is the fact Bacopa wrote from the section our chunk ran into:

The model is asked for facts that meet a few rules:

• Self-contained. A fact names what "it", "this" and "the consumer" refer to, so it makes sense alone.
• Every specific kept. Defaults, limits, versions, commands and steps stay. Length follows the section, so a long algorithm stays whole in one fact.
• Noise left out. Navigation, hooks, recaps, promotion and calls to action go. A section that holds only noise gives no fact, and a section with two separate subjects may give two.
• Code copied verbatim. A program checks every code block against the page's text and removes any it can't find. It drops keywords the page doesn't contain the same way.

This sits close to agentic chunking and to proposition retrieval, from the paper Dense X Retrieval, which indexes short single-claim statements. The difference is the unit. A proposition holds one claim, which is too small for a procedure or a table of limits. A fact holds a whole section.

Search runs on the distilled text

Bacopa searches a fact by its words and by its meaning. Words match the statement, questions, keywords and topic, plus the page's title, ranked by BM25. Meaning compares the query's embedding with the embedding of the statement and questions. The two rankings merge by reciprocal rank fusion. The body isn't searched, since the statement already says what it teaches.

The questions help most. A developer asks "does one bad message retry the whole batch?" and the page says "if one message in a batch fails, the entire batch is retried". The fact's third question sits between the two phrasings. Search returns the fact in full, with the page's brief, which says what the page covers and gives caveats on how far to trust it.

What it costs

• Model calls. Every page goes through a model before anyone can search it. Bacopa sends sections in batches of up to 8,000 tokens and runs the calls in parallel, so most pages need one call.
• Time. A saved page is ready within 3 minutes, or 30 for a long document, where chunking takes seconds.
• Trust. A model can drop a detail or misjudge what counts as noise. A program checks what it can, code and keywords, and an eval with a model as judge measures the rest, so a change to a prompt is judged by numbers. The page's text stays too, and an agent can ask for the passage a fact came from when the fact falls short.

When chunking is the better choice

Chunking wins when documents change every few minutes, when the corpus is too large to pass through a model, or when the answer has to quote the original wording, as in contracts. Bacopa's case is the opposite. A person saves pages one at a time, and agents read them again and again. Paying for a model once per page buys an agent a clean, self-contained fact every time it searches.