Exchange rate (FP share of priced error)
n/a
INSTRUMENT | Reliability testing
Acceptable Error Rate Calculator: The Reliability Bar Automation Needs
3 cited sources
Invert your own manual cost, automation cost, and consequence cost into the minimum reliability that makes automating pay, and see if a measured rate clears it.
A measured pass rate either clears a bar or it does not. This inverts your manual cost, automation cost, and the price of one error into that bar. Detection defaults to 0; the evidence is in what a silent failure costs when it looks like a success.
Split false positives from false negatives when they cost differently, and read the exchange rate alongside where to cut a confidence threshold.
Enter your costs to find the bar.
Minimum acceptable reliability
Not priced yet: see the note above.
Monthly failure budget
n/a
These are your own rates. We filled nothing in.
How the bar was built
| Build-up | Value |
|---|---|
| Automation saving per task (S) | n/a |
| Expected loss per error (L) | n/a |
| Monthly net saving at the measured rate | n/a |
Minimum reliability at 0.25x to 4x the consequence cost entered
| Consequence cost | Minimum reliability |
|---|---|
| 0.25x | n/a |
| 0.5x | n/a |
| 1x | n/a |
| 2x | n/a |
| 4x | n/a |
e* = S / L + manualErrorRate / 100, where S = manualCost - autoCost - checkCost and L = (detectionRate / 100) x reworkCost + (1 - detectionRate / 100) x consequenceCostHow?
How this is calculated
Automating a task saves S dollars against doing it by hand. Every error costs L, split between a caught error (rework only) and an escaped one (the priced consequence). The break-even error rate is the saving divided by the price of an error: above that rate, the errors cost more than the saving buys; at or below it, automating still pays. A stated manual error rate raises the maximum acceptable error rate, and lowers the minimum reliability bar by that same share, because a fair comparison prices hand work's own mistakes too.
Worked example. Manual cost $4.00, automated cost $0.15, a $0.05 check, a $2.00 rework cost, a $120.00 consequence, and 60 percent detection. S is $3.80 and L is $49.20, for a bar of 92.28 percent, about 1 error in 13 tasks. A measured 940 of 1,000 has a 95 percent interval of roughly 92.4 to 95.3 percent, entirely above the bar, so the measurement clears it.
Split mode. False positives and false negatives often cost differently and get caught at different rates. Splitting them prices each branch on its own detection rate, then blends the two by their share of all errors. The exchange rate reads the false-positive share of the blended price, off the two branch losses, never off the raw entered costs alone: a detection rate that differs between the two kinds moves the exchange rate away from a simple cost ratio.
No acceptable rate for everything. This page prices recoverable failure. Permission enforcement and prohibited actions carry no acceptable rate at any price: a hard invariant gets zero budget, not a number derived from a cost matrix.
Formula: e* = S / L + manualErrorRate / 100, where S = manualCost - autoCost - checkCost and L = (detectionRate / 100) x reworkCost + (1 - detectionRate / 100) x consequenceCost
Questions
My measured rate straddles the bar. Now what?
That is the honest answer at your sample size: the interval is not tight enough to say which side of the bar you are on. Collect more data, or put a confidence interval on the rate you already have with the pass-rate confidence interval calculator to see how much more you would need.
Where do the rework cost, consequence cost, and detection rate come from?
Paste them from the expected cost of failure calculator's own outputs: its per-caught figure is your rework cost, its per-escaped figure is your consequence cost, and its detected share is your detection rate. Each is per occurrence, never a period total.
Why does the exchange rate differ from the false-positive cost divided by the false-negative cost?
Because the exchange rate reads the two BRANCH losses, not the raw entered costs. A branch loss already blends in that kind's own detection rate, so two error kinds with the same entered costs but different detection rates still exchange at different rates.
What if hand work never actually errs?
Leave the manual error rate blank; it reads 0 and the bar is exactly S divided by L. The field exists for the honest case where hand work has its own error rate, priced at the same L as the automated path's mistakes.
Does volume change the bar?
No. There is no fixed cost in this model, so the minimum reliability is the same whether you run ten tasks a month or ten million. Volume only sets the monthly failure budget, the count of errors that bar allows; splitting that budget across error classes is what the error budget calculator does next.
How is this different from cost per successful task?
The cost-per-successful-task calculator prices what one success costs across every attempt it took to get there. This page asks the opposite question: given what a mistake costs, how unreliable can the automated path be before it stops paying for itself.
Sources
- Economic Evaluation of LLMs (pricing error in dollars per query, the framing this page inverts)
- The Foundations of Cost-Sensitive Learning (the cost-matrix threshold this page inverts)
- A simple, step-by-step guide to interpreting decision curve analysis (the threshold probability read here as an exchange rate)