Topic
RAG & retrieval
Retrieval as a reliability problem: chunking, faithfulness, and the failure modes that live between the index and the answer.
Research
- Analysis
Is grep all you need? Agentic search vs vector retrieval
The PwC paper behind the grep debate: lexical search won every inline pairing on LongMemEval, lost five of ten once results moved to files, and the harness shifted accuracy as much as the retriever.
- Analysis
RAG evaluation metrics, and what each one certifies
Each RAG evaluation metric certifies one narrow property. What context precision, context recall, faithfulness, groundedness, answer relevancy and retrieval drift answer, and what each cannot see.
- Analysis
Agentic RAG architecture, and where each pattern breaks
Agentic RAG wraps retrieval in an autonomous agent that plans, routes, and self-corrects. Here is the architecture, the failure modes each pattern adds, and when classic RAG is the safer call.
- Analysis
Where RAGAS wins at RAG evaluation, and where it stops
A positioning read on RAGAS for RAG evaluation: its real strengths, a limitation its own authors flag, and the rigor gaps around confidence intervals, significance, and propagation-aware attribution.
- Analysis
How to evaluate a RAG pipeline beyond a single score
Evaluate a RAG pipeline by scoring retrieval and generation separately, putting a confidence interval on every metric, and attributing each failure to the stage that produced it.
- Analysis
RAG pipeline failure modes, and the gate that stops each one
RAG pipeline failure modes are containment failures at the retrieval-to-generation boundary: five modes mapped to how each propagates, its detection signal, and the gate that holds it.
Terms
-
Answer relevancy
Answer relevancy is the degree to which a generated answer responds to the question actually asked, scored from the question and the answer alone. It carries no check on whether the answer is true, so it is read beside a grounding score rather than on its own.
-
Context precision (RAG evals)
Context precision is the share of the chunks a retriever returned that were relevant to the question, scored over the retrieved set rather than over the corpus, and reported as a mean across a query set with an interval.
-
Context recall (RAG evals)
Context recall is the share of the evidence a correct answer needs that the retriever actually returned, scored per question against a reference for that question rather than against the whole corpus. It sets the ceiling every generation metric downstream is measured under.
-
Faithfulness (RAG evals)
Faithfulness in RAG evaluation is the proportion of an answer's claims that can be inferred from the retrieved context, computed as verified statements over extracted statements. It scores the generation step against the passages the model was actually shown.
-
Groundedness (LLM answers)
Groundedness is the share of an answer's factual claims that trace to an identified source a reader could open and check, judged claim by claim against the sources a system named or was handed. It is wider than faithfulness, which scores only against retrieved context.
-
Retrieval drift
Retrieval drift is the loss of retrieval quality in a live pipeline over calendar time, read as the change between what a frozen query set returns today and what the same queries returned at an earlier baseline. The quantity is a difference, so it carries an interval.