LatentEval

INSTRUMENT | Propagation & containment

Expected Cost of Failure Calculator for AI Agents

1 cited source

Price a failure rate in money: what gets caught, what escapes, and what correction and rework savings are worth, split by failure class.

A failure rate is not a cost until you price what gets caught, what escapes, and what correction takes. Grounded in what a silent failure costs when it looks like a success and how errors compound down a chain.

Savings mode reads the same arithmetic backward: cut a class's rate and see what it is worth, next to what one acceptable result costs and the bar your reliability needs to clear.

Expected loss or savings
Volume and period

Blank or zero is rejected: it is the divisor for per-task cost.

Check this value.

Check this value.

Correction pricing

Money per hour. Blank holds the money region back; it is not an error.

Check this value.

How much costlier a late fix is than an early one. Blank holds the money region back.

Check this value.

Exposure time only. No money is priced from this field.

Check this value.

USD per correction attempt. Only applies to caught occurrences: you cannot auto-fix what you never detected.

Check this value.

Savings mode

What the intervention itself costs, for this period.

Check this value.

Failure classes

Label Share Detect Hours Reversal Downstream Remediation Reduction Actions

Check this value.

Check this value.

Check this value.

Occurrences
9,000
Caught before delivery
5,850
Escaped
3,150
Detected share
65.0%
Exposure hours
0 hrs

Enter a labor rate and a delay multiplier to price these occurrences. A delay multiplier of 1 means no penalty for a late fix.

Expected loss per month

Not priced yet: see the note above.

Per task

n/a

Escaped share of loss

n/a

Per failure class

Failure classOccurrencesCaughtEscapedCaught costEscaped costClass lossShare of loss
Wrong data extracted 6,000 3,000 3,000 n/a n/a n/a n/a
Skipped a required check 3,000 2,850 150 n/a n/a n/a n/a
(unlabeled) 0 0 0 n/a n/a n/a n/a

Savings mode read on the same numbers

Savings modeValue
Savings n/a
Loss after savings n/a
Net after change cost n/a
Break-even change-cost share n/a
Export

classLoss = caught x caughtCost + escaped x (M x correctionCost + downstream + remediation)How?

How this is calculated

Every failure class splits into what gets caught before delivery and what escapes. A caught occurrence costs correction labor, a reversal, and any auto-fix attempt. An escaped occurrence costs the same labor and reversal, but the delay multiplier scales that internal correction cost, plus whatever downstream impact and customer remediation the reader already priced for the escaped case. The multiplier never touches downstream or remediation a second time, since those figures already price the late, escaped outcome directly.

The per-caught and per-escaped figures above are per-occurrence dollars, never a period total. The per-escaped figure is the cost of one failure that reaches the customer after every check you run; paste it into the acceptable error rate calculator as its consequence cost, and the per-caught figure as its rework cost. Whether running that check is worth it in the first place is what the review threshold optimizer weighs, once both figures are in hand.

The delay multiplier. Boehm and Basili's measurement of defect-cost growth puts the ratio at about 100 to 1 on large projects and closer to 5 to 1 on small ones. This page asks for that ratio as a field rather than assuming one, because no single number fits every team's pipeline.

Detection hours. The hours to notice an escaped failure feed only an exposure figure, in hours. No money is priced from detection time here; that is the reader's own correction-labor field, priced once the failure is found. The same hours feed the error budget calculator's burn-rate window.

Savings mode. Cutting a class's failure rate by a stated percentage removes that share of its loss. The model does not change between modes, only the reading: savings is the same arithmetic looked at backward, against what a change to fix it would cost. Once you know what a failure costs, the acceptable error rate calculator inverts these same figures into the reliability bar automating would need to clear.

Formula: classLoss = caught x caughtCost + escaped x (M x correctionCost + downstream + remediation)

Two worked examples

A flat rate with no detection. 30,000 tasks a month, one failure class at a 30 percent share, no detection, a delay multiplier of 1, and a flat $10 remediation cost with everything else zero. That is 9,000 occurrences a month, all escaped, at $90,000.00 total and $3.00 per task. The figures are illustrative and reproduce no source this page names.

One model read twice. 12,000 tasks a month, one class at a 5 percent share, no detection, and a $20 remediation cost gives 600 occurrences and a $12,000.00 loss. Cutting that class's rate by 40 percent removes 40 percent of its loss: $4,800.00 in savings, leaving $7,200.00. The numbers are this page's own illustration.

Sources

  1. Software Defect Reduction Top 10 List (the measured 100:1 large-project and about 5:1 small-project defect-cost multiplier this page asks the reader to set as a delay multiplier)IEEE Computer, January 2001 (Boehm and Basili) Retrieved