Context recall (RAG evals)

Context recall is the share of the evidence a correct answer needs that the retriever actually returned, scored one question at a time against a reference for that question. It runs in the opposite direction from context precision, which divides by what came back; recall divides by what the answer required, so the two move independently and neither one predicts the other.

Recall sets the ceiling for everything downstream, because generation cannot ground a claim on a passage it never saw, and that is the retrieval miss sitting at the top of the RAG failure taxonomy. Score it before you spend a week tuning the generator, then re-score it on a schedule, since an index that keeps growing quietly changes what the same query returns and that is retrieval drift working on your numerator.

Because the denominator is everything a correct answer needed, and nobody can enumerate that from the corpus alone, a context recall figure is only as good as the reference it was scored against. A per-question score is a proportion, so the suite mean inherits binomial uncertainty and wants an interval around it rather than a lone decimal. The reference carries a second uncertainty no interval covers, so name it beside the number, the way you would publish the denominator behind any rate.

How to calculate context recall

Take one question, write down what a correct answer to it needs, count how much of that appears in the chunks the retriever returned, and divide. The three routes in common use differ only in how the reference gets written down. Chunk IDs are cleanest: with a gold set naming the IDs each question needs, count the gold IDs present in the retrieved set over the gold IDs total, and pay for it in labeling. Gold passages without IDs put string similarity in place of identity, at the price of a matching threshold you now have to defend. A gold answer on its own leaves the claim-level route, where you decompose that answer into claims and check each one for support in the retrieved context, which is how RAGAS computes its LLM-based context recall and where our reading of what RAGAS certifies as an LLM judge applies in full.

Each route hands you a different denominator and a different score for the same retriever, and the last one puts a judge inside what reads as a retrieval metric. Audit that judge the way you would audit any other: measure its agreement with a human on a sample before you trust a recall number it produced, and use chance-corrected agreement, since two raters who both say yes most of the time will agree a lot by accident.

Report the suite mean, and pick the interval to match how you scored. Binarised, where a question passes only when retrieval returned everything the answer needed, the figure is a proportion and the pass-rate interval calculator covers it. Kept fractional, per-question scores aren’t Bernoulli draws, so resampling the query set is what gets you an interval that means something. Underneath all of it sits one log line: the retrieved chunk IDs stored per request beside the response, without which recall can only be recomputed by replaying retrieval against an index that has since moved.

One property of recall shapes every experiment you run on it. Recall is a non-decreasing function of the number of documents retrieved, as the standard information retrieval text puts it, so raising top-k raises recall by construction and a recall gain reported without its k describes nothing anyone can check. What raising k costs is context budget, which the chunk size and context budget calculator turns into a share of the window per retrieval. An agent that searches iteratively has no single top-k, so its recall is scored over the union of everything it pulled across the trajectory, one reason grep-style agentic search and embedding retrieval resist a single comparison.

Context recall vs context precision

Context precision is the share of the retrieved chunks that were relevant to the question, computed over what the retriever returned. Context recall divides by what the answer required. The pair separates two failures a single retrieval score blends together, and the same system can fail either way.

Push top-k up and the first divergence opens. A k of 50 sweeps in the passage the answer needed along with dozens nobody will read, so recall climbs toward one while precision falls toward the reciprocal of k, and the generator now has to find one relevant passage in a window mostly filled with near misses.

Tighten the retriever and it opens the other way. A similarity threshold strict enough that only two chunks clear it can return two relevant chunks, a precision of 1.0, on a question whose evidence was spread across four documents. Recall of 0.5 with perfect precision produces a confidently partial answer that every grounding check waves through, because everything the answer said really was supported by the half that got retrieved.

Compute them over the same query set at the same k, so a retrieval change that trades one away for the other shows up as a trade.

Context recall vs recall@k

Recall@k in search evaluation is the fraction of the documents judged relevant to a query that appear in the top k results, with the judgments coming from a relevance file human assessors built for that query set. Information retrieval has owned the word for decades and owns it legitimately, and a search team already computing recall@k has a working pipeline worth keeping.

The denominators differ, and that difference is the whole disagreement. Recall@k divides by documents an assessor called relevant to the query; context recall divides by the evidence one specific reference answer required. A document can be squarely on topic and contribute nothing the answer needed, while a single sentence buried in a document an assessor ranked marginal can be the only source of a required fact. The gap opens widest on multi-hop questions, where a topical judgment file looks satisfied the moment any relevant document arrives, so recall@k reads high while context recall reads low.

Neither reading supports a single benchmark number for retrieval quality, because recall depends on k, on the corpus, on the judgment procedure and on the question mix. Quote it with its k, its reference type and its query set, hold all three fixed across comparisons, and report each one with its interval.

Faithfulness scores perfectly on a context that recall left half empty

Faithfulness scores the claims in the generated answer against the retrieved context, asking whether the model asserted anything the context does not support. Context recall runs the same attribution machinery over a different numerator, the claims in the reference answer, and asks whether the context contained them in the first place. The judge is the same and the context is the same, and the question being put to them is the opposite one.

The two of them separate hardest in one direction. An answer can post a perfect faithfulness score on a context missing most of what the question needed, and the result is a well-grounded wrong answer: every sentence traceable to a retrieved passage, while the passage that would have corrected it was never retrieved. A groundedness check passes, answer relevancy passes, and the answer is still wrong. The reverse case is ordinary hallucination, where recall of 1.0 with faithfulness far below it says the evidence sat in the window and the generator went around it.

Read recall first, because it bounds what a faithfulness score is worth on that question. Faithfulness measured on a half-empty context describes the model’s discipline and doesn’t tell you whether the answer is right.

Recall and precision turn a vague sense that retrieval is weak into two numbers with separate fixes. Each of the six RAG metrics answers one question and declines several others, and the map of which metric certifies what keeps them from being read as interchangeable, while the fuller method of scoring retrieval and generation separately sits in our method for evaluating RAG pipelines without a single blended score. A recall figure carries the reference it was scored against, so a team that cannot fund and maintain a gold set should take the claim-level route with a judge it has audited. Quote the reference type beside the number, because a recall of 0.8 scored against decomposed claims may not be the same quantity as a recall of 0.8 scored against gold chunk IDs.