LatentEval

INSTRUMENT | Propagation & containment

Cost per Successful Task Calculator: What One Acceptable Result Costs

2 cited sources

Turn a per-attempt price and a pass rate into the cost of one acceptable result, with retries, failed attempts, review and the fallback path all priced into the total.

A two-cent attempt that works a third of the time costs about six cents per result. This page prices that gap from your own numbers, with retries, failed attempts, review and the fallback path all in the total.

It rests on our measurements of what refused and retried work adds to a bill and of how to measure agent reliability.

What one attempt costs
Price an attempt

USD per attempt. What one try costs you, all in. Leave it empty to see the reliability figures alone.

Check this value.

How often it works

How often one attempt gives a result you would ship. 0.7 or 70 both mean 70 percent.

Check this value.

I have run this and counted

Enter what you measured and the rate above is replaced by it, with a 95 percent interval priced beside the point estimate.

How many runs gave a result you would ship.

Check this value.

How many runs you counted in total.

Check this value.

What happens when it does not

The first try plus every retry you allow.

Check this value.

Empty means every retry is a fresh roll of the same odds. 0 means a task that failed once never passes. 0.7 or 70 both mean 70 percent.

Check this value.

As a multiple of a passing attempt. 1 when they cost the same. One measured study puts failures at 1.27 times the cost.

Check this value.

USD per task that reaches it. What the escape hatch costs: a bigger model, a person, a manual path.

Check this value.

How often the fallback gives a result you would ship. 0.7 or 70 both mean 70 percent.

Check this value.

Who checks it

USD per reviewed item. A person's time, or a second model.

Check this value.

How much of the work gets checked. 0.2 or 20 both mean 20 percent.

Check this value.

Review counts

Tasks that end acceptable

70.0%

1.00 attempts per task at these settings.

Cost per attempt

not priced

Cost per successful task

not priced

These are your own rates. We filled nothing in.

Enter your cost per attempt to price this.

30.0% of tasks use up every attempt without an acceptable result.

LinePer task attemptedPer successful task
Running the attempts not priced not priced
Review not priced not priced
Fallback not priced not priced
Total not priced not priced

What the same setup costs at other pass rates.

ScenarioPass rateEnds acceptableCost per successful task
10 points lower 60.0% 60.0% not priced
5 points lower 65.0% 65.0% not priced
Your rate 70.0% 70.0% not priced
5 points higher 75.0% 75.0% not priced
10 points higher 80.0% 80.0% not priced
Export

cost per success = (attempts + review + fallback) / P(acceptable)How?

How this is calculated

The divisor is successes. A price per attempt divides your spend over everything you ran. This divides it over the runs that produced something you would ship. At a 33 percent acceptance rate, one acceptable result costs three times the sticker price before anything else is counted, and that multiple is the whole point of the figure.

What a task costs. Four lines. The attempts, where a failed one can cost more or less than a passing one. The review, either once per task or once per attempt. The fallback, charged only to the share of tasks that use up every attempt. Those three make the cost of one task attempted; dividing by the chance a task ends acceptable makes the cost of one result.

Retries are the assumption to watch. By default a retry is a fresh roll of the same odds, which is the cheapest reading of a retry. Real failures repeat: a hard task that failed once tends to fail again. So the page always computes the other end, where no retry ever helps, and shows it beside the headline whenever you allow more than one attempt. At three attempts on a 33 percent rate the two ends are 2.3 times apart. If you want to model the retry decision itself rather than price it, the retry and fallback comparison does that, and the correlated-failure simulator shows what shared causes do to a run of steps.

Where the rate comes from. Type one, or enter what you counted and the page derives it with a 95 percent interval and prices both ends. An interval on a rate becomes an interval on the money, and a cost quoted with no interval on a rate measured over fifty runs is a cost quoted more precisely than it is known. If your task is a chain of steps rather than one call, the end-to-end rate is a product of the per-step rates, and that product is the number to bring here. To price the eval run that measures the rate in the first place, the eval budget calculator takes it from the other end.

Rounding. Everything computes at full precision and rounds only for display. Money shows two decimals, or four below a dollar, because a real sub-cent cost printed as $0.00 is worse than useless. Rates show one decimal, expected attempts two, and the multiple one.

Honest limits. This is arithmetic on numbers you supply, and it inherits their quality. It has no opinion on how much review to buy, and it says nothing about what a wrong answer costs once it ships: that loss sits outside this total and is usually the larger number. A rate measured on forty tasks is a rate measured on forty tasks, whatever this page prints beside it.

Formula: cost per success = (attempts + review + fallback) / P(acceptable)

Questions

Why not just divide the cost per attempt by the pass rate?

That is exactly what this does when nothing else is entered, and the answer is the same to the last decimal. The three things it hides are the ones that move the number: a failed attempt that costs more than a passing one, retries whose odds are not independent, and every dollar of review and fallback around the model call. Leave those at their defaults and you get the short form. Fill them in and you get your bill.

What counts as a successful task?

Whatever you would ship without touching it. That threshold is yours to set and it belongs upstream of this page: a test suite that passes, a reviewer who accepts, a schema that validates. The figure is only as meaningful as that line is stable, so write it down before you measure a rate against it.

Should a failed attempt really cost more than a passing one?

Often, yes. A failure tends to wander: more steps, longer traces, more retries inside the call, and a timeout costs the full budget before it gives up. One study of agent runs measured failures at 1.27 times the cost and 2.77 times the cycles of a success. The default here is 1, so the page never inflates your number on its own. Measure your own ratio if the failures are expensive.

Why does the page show a second, higher figure when I allow retries?

Because the headline assumes each retry is an independent roll, and that is the optimistic end of a range rather than a neutral middle. The second figure is the other end: a task that failed once never passes, so the extra attempts add cost and nothing else. Your real system sits between them. If you have measured how often a retry actually rescues a failure, enter it as the retry acceptance rate and both figures collapse toward the truth.

Where do the preset prices come from?

A person read them from each vendor's own pricing page on the date shown beside every rate. They are a convenience, not an authority, and a price can change the day after it is read. Any number you type wins over ours, and editing a filled field turns the stamp back to your own number. Presets reach this page through per-token mode only, because published prices are per token and a cost per attempt is yours.

Does this include what a bad answer costs after it ships?

No. Everything here is the cost of producing an acceptable result. The expected loss from a wrong answer that nobody catches is a separate calculation, and for most teams it is the larger of the two. Treat this total as the floor of the real number.

What questions sit next to this one?

Four, and none of them is this one. What one uncaught failure costs, once it reaches whoever it reaches. How good the system has to be before that loss is acceptable, which is the same arithmetic read backward into a target error rate. Whether the review that catches those failures pays for itself, and where to set its cutoff. And how much failure a month of running can absorb before the budget for it is gone. This page answers the price of a success, which every one of those four needs as an input.

Where do the token counts come from in per-token mode?

From your own prompt, counted rather than estimated. A cost per attempt built from a guessed token count is a guess with two decimal places on it. Take a real request and a real response, count both, and enter those. If the two prices differ by a factor of five, as most vendors price them, the output count moves your number more than the input count does.

Sources

  1. Beyond Accuracy: a multi-dimensional framework for evaluating enterprise agentic AI systems. Defines cost per success as total cost divided by the number of successful tasks, which is the figure this page computes, and reports agents optimized for accuracy alone costing 4.4 to 10.8 times a cost-aware alternativearXiv Retrieved
  2. Evaluating LLM agents on automated software analysis tasks. Section 4.4 measures failed runs against successful ones on the same workload: 2.77 times the cycles, 4.07 times the duration, 1.27 times the cost and 1.76 times the retries. That measurement is why this page prices a failed attempt separately from a passing onearXiv Retrieved