Wilson and Clopper-Pearson Pass-Rate Confidence Interval Calculator

This tool reports a Wilson score interval by default: the range that brackets the true pass rate p behind an observed c passes in n independent runs, at the confidence level you pick. It also shows the Clopper-Pearson exact interval and the Wald normal approximation on the same inputs, so you can see where they agree and where they part.

Every method assumes each run is an independent Bernoulli trial at one fixed pass probability. That is the binomial model behind an eval confidence interval. The point estimate c/n is one draw from that distribution; the interval is the width around it that the evidence supports.

Wilson score interval (the default). With z the two-sided normal quantile for your confidence level, the interval centers and margins are:

center = (p̂ + z²/2n) / (1 + z²/n)

margin = (z / (1 + z²/n)) · √(p̂(1−p̂)/n + z²/4n²)

Worked example, c = 30, n = 100, 95% (z ≈ 1.9600, p̂ = 0.30, z² = 3.8415): center = (0.30 + 3.8415/200) / (1 + 3.8415/100) = 0.31921 / 1.03841 = 0.3074; margin = (1.9600 / 1.03841) · √(0.30·0.70/100 + 3.8415/40000) = 1.8875 · √0.0021960 = 1.8875 · 0.046862 = 0.0884. The interval is 0.3074 ± 0.0884, or 21.9% to 39.6%. Wilson stays inside 0% and 100% and returns a real width even at a 0% or 100% observed rate, which is why it is the default here.

Clopper-Pearson exact interval. The bounds are Beta quantiles, so the interval has at least the nominal coverage (it errs wide rather than narrow):

lower = BetaInv(α/2; c, n−c+1), upper = BetaInv(1−α/2; c+1, n−c), α = 1 − confidence

For the same c = 30, n = 100, 95% case: lower = BetaInv(0.025; 30, 71) = 0.2124, upper = BetaInv(0.975; 31, 70) = 0.3998, or 21.2% to 40.0%. Report this when you need a guaranteed-coverage bound and can accept a wider interval.

Wald normal approximation. The textbook bound is p̂ ± z·√(p̂(1−p̂)/n), which for c = 30, n = 100, 95% gives 0.30 ± 0.0898, or 21.0% to 39.0%. It is shown for contrast and carries a guard. The Wald interval collapses to a single point at a 0% or 100% observed rate, and it can fall outside 0% to 100%. This tool flags it as unreliable when either the observed passes or the observed failures fall below 10, the success-failure count condition. That floor of 10 is a common convention; Brown, Cai and DasGupta (2001) show Wald coverage stays erratic even at large n for rates near 0 or 1, so treat the floor as a minimum rather than a guarantee.

What the confidence level means, and what it assumes. A 95% two-sided interval is a frequentist statement about the procedure: across many repeated suites, about 95% of the intervals it builds would contain the true rate. It is not a 95% probability that this one interval holds the true rate. The binomial model also assumes runs are independent and identically distributed. Correlated seeds, a shared prompt, or a heterogeneous task mix break that assumption, and the real uncertainty is then wider than any bound here. Add the run-to-run spread from independent seeds on top of the task-set width this tool computes.

Formula: Wilson: (p̂ + z²/2n ± z·√(p̂(1−p̂)/n + z²/4n²)) / (1 + z²/n)

See Sources below