LatentEval
Topic

Evidence and reproducibility calculators

6 calculators | 3 analyses

What has to be written down for a number to be checkable later: the card behind a result, the manifest behind a re-run, and the claim each piece of evidence is actually evidence for.

The method

What these numbers mean

An eval number that nobody can re-run is a claim, not evidence. The instruments below are about the part that survives the run: what was recorded, whether someone else could reproduce it, and which specific claim each artifact is actually evidence for.

What you haveThe instrument
An eval number about to go into a deck, with no check on whether it is quotablerun the checklist before citing it
A result and no record of the conditions it was measured underwrite the evaluation card
A card or a manifest, and a doubt about whether anyone could re-run itcheck it against the reproducibility profile
A set of claims going into a deployment decision, and a pile of artifacts beside themregister each claim against its evidence
A launch decision, with evidence to assemble into a readout for sign-offbuild the launch readout

The method these instruments apply is written out in the evidence quality framework, which defines the per-claim status vocabulary and the dimensions each claim is assessed against. This family rests on our own work on the four ways an eval number lies and how to report a judge-scored evaluation honestly.

The fork here is whether a gap is absent or recorded. A card with a null temperature and a stated reason is a different object from a card that never had the field. Only the first one tells a reader what to go and find out.

In this cluster

Tools in this topic

Tool | Evidence and reproducibility

Evaluation Card Generator: Markdown and JSON on a Published Schema

Write one evaluation down so a second person can read the number correctly, or run it again. Fill the form, export a Markdown card and a JSON card that validates against a schema published here.

Tool | Evidence and reproducibility

Claim-Evidence Matrix: What Backs Each Deployment Claim

One row per deployment claim, eleven evidence dimensions from relevance and representativeness to independence and applicability, with named rate-down reasons and residual deficits.

Tool | Evidence and reproducibility

Is this eval number ready to quote?

A short check before an eval number goes into a deck or readout. The page returns quotable, quotable with caveats, or not yet, with the sentences to paste beside the number.

Tool | Evidence and reproducibility

Write the eval readout for a launch decision

Record a launch decision with its evidence: each figure's interval and source, the segments, residual risk and what was not tested. Export it for sign-off or paste a summary.

Tool | Evidence and reproducibility

Reproducibility Checklist and Manifest Validator

Check whether your eval write-up has what someone else needs to rerun it, or capture evidence from a production failure before it is lost. Three modes: checklist, manifest validator, incident record.

Tool | Evidence and reproducibility

Eval results auditor: find errored, empty and unreadable rows

Reads a results file or pasted rows, classifies each row that got no verdict (errored, NaN, empty, unparsed grade), and recomputes the pass rate with unscored rows counted as failures.