Index entries
1,500
INSTRUMENT · EVAL
RAG Chunk Size Calculator: Index Size and Context Budget
Size a RAG index from its chunking plan: from chunk size, overlap, and top-k, get the vector count, overlap storage inflation, and context-budget share.
Context budget used
2%
Full sizing readout
Chunk tokens means the effective chunk size, min(chunk, average document length): a document shorter than one chunk becomes a single smaller entry.
| Quantity | Value | How it is derived |
|---|---|---|
| Effective stride | 435.2 tokens | chunk × (1 − overlap) |
| Chunks per document | 3 | 1 + ⌈(avg doc − chunk) ÷ stride⌉ |
| Index entries (vectors) | 1,500 | chunks per doc × documents |
| Retrieved context per query | 2,560 tokens | top-k × chunk tokens |
| Context budget used | 2% | retrieved ÷ budget |
| Budget left per query | 125,440 tokens | budget − retrieved |
| Tokens stored in index | 768,000 tokens | index entries × chunk tokens |
| Storage inflation | 1.28× | stored ÷ corpus tokens |
stride = chunk × (1 − overlap)How?
How this is calculated
This calculator turns a chunking plan into the two numbers that decide whether it is affordable: how many vectors land in the index, and how much of the model's context budget one retrieval consumes. It is deterministic token arithmetic, so it carries no confidence interval. Retrieval quality, the recall and precision a chunk size actually buys, is a measured and uncertain quantity that you read off an eval run, covered in how to evaluate a RAG pipeline. This tool sizes the consequences of a chunk size; it does not rank chunk sizes.
The chunking model. Chunking is treated as a fixed-size sliding
window that advances by a stride of chunk × (1 − overlap). A
document of avgDoc tokens yields one chunk when it fits inside a single
window, and otherwise 1 + ⌈(avgDoc − chunk) ÷ stride⌉
chunks. This matches a fixed-token chunker; a sentence- or structure-aware splitter
lands nearby, not exactly. Overlap never crosses a document boundary, so when a
document count is known the chunks are counted per document from the average length
and the total is an estimate for a corpus of uneven documents.
Worked example (the default: 500 documents of 1,200 tokens, chunk 512, overlap 15%, top-k 5, budget 128,000):
stride = 512 × (1 − 0.15) = 435.2 tokens
chunks/doc = 1 + ⌈(1200 − 512) ÷ 435.2⌉ = 1 + ⌈1.581⌉ = 3
index entries = 3 × 500 = 1,500 vectors (one embedding per chunk).
retrieved/query = 5 × 512 = 2,560 tokens, which is
2,560 ÷ 128,000 = 2% of the budget, leaving 125,440 tokens for the
prompt and the answer.
tokens stored = 1,500 × 512 = 768,000, so storage inflation is
768,000 ÷ 600,000 = 1.28× the raw corpus. In the long-document
limit that ratio approaches 1 ÷ (1 − overlap), the token cost
overlap adds to the index.
What the budget share leaves out. Retrieved tokens are counted as
top-k full chunks, so the figure is a slight over-estimate: it ignores
the de-duplication of overlapping tokens when adjacent chunks come back together.
It also counts only retrieved context. Prompt scaffolding, tool definitions, the
conversation so far, and the generated answer draw on the same budget, so treat the
share as a floor on what retrieval consumes, not the whole call. When a document is
shorter than one chunk, its single entry is the size of the document, so the
per-query and stored token counts use min(chunk, avgDoc).
Formula: stride = chunk × (1 − overlap)
Questions
What chunk size should I pick?
This tool does not name an optimal chunk size, because there is no size that is best in the abstract. The right size depends on your documents, your embedding model, and the questions you retrieve for, and it is settled by measuring recall and precision on a labelled set, not by arithmetic. Use this calculator to see what a candidate size costs: the vector count, the storage inflation from overlap, and the slice of the context budget each retrieval takes. Then read how to evaluate a RAG pipeline to measure which size actually retrieves the right passages.
Why does the index hold more tokens than my corpus?
Overlap. Each chunk after the first repeats a share of the previous chunk, so a
long document is stored more than once across its chunks. Storage inflation is the
stored tokens divided by the corpus tokens, and it climbs toward
1 ÷ (1 − overlap) as documents get long relative to the chunk:
15% overlap tends toward about 1.18×, 25% toward about 1.33×. Documents
shorter than one chunk add no duplication, so a corpus of many short documents
shows an inflation near 1× whatever the overlap setting.
How is the context budget share computed?
It is top-k × chunk tokens ÷ context budget, the retrieved
context as a fraction of the tokens the model accepts per call. When top-k asks
for more chunks than the index holds, the query returns every chunk and the share
is computed on that smaller number. The share counts retrieved context only, so
add the prompt, tool definitions, prior turns, and the answer on top before
deciding a plan fits.
Sources
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (defines the RAG retrieval setting)
- Lost in the Middle: How Language Models Use Long Contexts (why retrieved-context budget is a real constraint)
- RAGAS: Automated Evaluation of Retrieval Augmented Generation (measuring retrieval quality a chunk size buys)