LatentEval

TOOL | Propagation & containment

Per-correct-answer cost of a two-tier router, misroutes included

Does the router still save money once its own mistakes count? Enter per-tier prices, pass rates and misroute rates to see cost per correct answer, the quality gap, and when routing stops paying.

Enter per-tier prices, pass rates and your model router's misroute rates. The calculator returns cost per correct answer for the routed mix against an all-premium baseline, the extra failures per 1,000 tasks, and the misroute rate at which the saving disappears, grounded in our pre-registered routing study and in pass rates measured past a single run. The cost-per-successful-task calculator prices one path; this page prices the mix.

What one attempt costs on each tier
Pricing

Price each attempt as a flat rate or derive it from input and output token counts.

USD per attempt. Leave both tier prices blank to see dispatch shares and quality figures without cost.

Check this value.

USD per attempt. Fill both tier prices or leave both blank.

Check this value.

How the traffic splits

Share of your traffic that the cheap tier handles acceptably: a property of the work, set by the task distribution. Enter 0.7 or 70 for 70%.

Check this value.

Probability the router sends an easy task to the premium tier. Costs money, gives up no correctness under the default assumption.

Check this value.

Of the tasks that needed the premium tier, the share the router sent to the cheap one. Measure it by auditing a sample of routed traffic.

Check this value.

I have audited my routed traffic

The measured rate replaces the one above, with a 95 percent interval run through the whole model at each bound.

Of the audited tasks that needed the premium tier, how many did the router send to the cheap one.

Check this value.

Total audited tasks that needed the premium tier: the denominator for the misroute-down rate.

Check this value.

How often each tier works

Acceptance rate when the cheap tier handles work it was meant for. A 'correct answer' on this page is the host system's acceptable result: shippable without edits.

Check this value.

Acceptance rate when the cheap tier handles work that needed the premium tier. Measure this if you have not.

Check this value.

Acceptance rate when the premium tier handles the work it was meant for.

Check this value.

Leave blank to assume the premium tier does no better on easy work than the cheap tier: our routing study's finding on short checkable tasks.

Check this value.

Routed mix pass rate

87.2%

Prices missing | Gap 1.4 pp | 14 extra failures per 1,000

Prices missing

The dispatch shares, pass rates and quality gap compute from the defaults. Add each tier’s price to see what a correct answer costs.

Cost figures are suppressed. The dispatch shares, blended pass rates and quality gap are live at these inputs.

The all-cheap row shows what the work would cost without a router.

ArmPass rateCost per attemptCost per correct answer
Routed mix 87.2% not priced not priced
All-premium baseline 88.5% not priced not priced
All-cheap floor 79.5% not priced not priced

Saving per correct answer

not priced

Saving per attempt

not priced

Divides by every attempt, including failures.

Share to cheap tier
67.5%
Share to premium tier
32.5%
Quality gap
1.4 pp
Extra failures per 1,000 tasks
14 tasks
Router discrimination
75.0%

Youden's J for the router as a binary classifier: the sum of the two correct-routing probabilities minus one.

These figures are a two-tier illustration at the default rates. Replace every rate and price with your own numbers.

Export

cost per correct answer = blended cost per attempt / blended pass rateHow?

How this is calculated

The four cells. A router assigns each task to one of two tiers. Each task either belongs on the cheap tier or needs the premium tier; the share that belongs on cheap is the easy-task share, b. The router errs in two directions: up, sending an easy task to the premium tier (costly, never wrong under the default assumption), or down, sending a hard task to the cheap tier (cheap, wrong more often). Every task lands in exactly one of the four resulting cells, and the four traffic shares sum to one.

Dispatch shares. The share of traffic the cheap tier receives is the correctly routed easy tasks plus the mistakenly routed hard tasks. Its complement goes to premium. These two numbers tell you what the router does to your traffic without needing any price or pass rate.

Blended pass rate. Each cell contributes its traffic share multiplied by the tier's pass rate on that kind of work. The routed mix's pass rate is the weighted sum across all four cells. Compare it against the all-premium rate to see the correctness the routing gives up. The gap can reach zero or turn negative: our routing study measured this on short checkable tasks where the premium tier's refusal rate was higher, and the page carries that finding as its default assumption for the premium tier's pass rate on easy work. Put an interval on any of the four rates with the pass-rate CI calculator before entering them here.

Cost per correct answer. Divide the blended cost per attempt by the blended pass rate. The page computes this for three arms: the routed mix, an all-premium baseline (every task sent to the premium tier) and an all-cheap floor (every task sent to the cheap tier). The cost-per-successful-task calculator derives the same ratio for a single path through one tier; this page sums the paths a router produces.

The saving, stated twice. The per-attempt saving is the premium price minus the blended routed price per attempt. The per-correct-answer saving replaces those with cost per correct answer. The page shows both. The per-attempt figure divides by total tasks; the per-correct-answer figure divides by tasks that pass.

The quality gap. The all-premium blended pass rate minus the routed blended pass rate, converted to extra failures per 1,000 tasks so the quantity has a unit a builder can picture. Turn those failures into a dollar figure with the rework and downstream cost page, or test whether the gap sits inside your tolerance on the error-rate threshold page.

Break-even misroute-down rate. Both cost per attempt and pass rate are affine functions of the misroute-down rate, so cost per correct answer equals the all-premium baseline at one closed-form crossing. Which direction erases the saving depends on the inputs: the page states the direction beside the crossing. When the inputs reverse the relationship, the saving grows with the misroute-down rate and is gone below the crossing, not above it. The page shows the value when it falls inside zero to one and suppresses it with its reason otherwise. One figure to carry to a budget conversation.

The interval, and the boundary. When the audited-sample pair is filled, a 95% Wilson interval on the misroute-down rate drives two re-runs of the full model, one at each bound. The higher bound sends more traffic to the cheap tier, producing a cheaper cost and a larger quality gap: the two move together. This page prices an up-front split where every task runs once on the tier it was assigned. A cheap-first policy that falls back to the premium tier after a detected failure is a different instrument; the retry and fallback comparison already prices the recovery. Delays, backoff and jitter belong to the backoff and drain planner.

Formula: cost per correct answer = blended cost per attempt / blended pass rate

Questions

What if I have no measured misroute-down rate?

Audit a sample of your routed traffic: pull the tasks the router sent to the cheap tier, have the premium tier re-grade them, and count the ones whose quality dropped. That count over the sample total gives you a measured rate you can type in here. Fill the audited-sample fields and the page puts a Wilson interval on it for you, or compute the interval first with the pass-rate CI calculator.

How is this page different from the retry strategy optimizer?

This page prices an up-front split: traffic is routed to one tier before any attempt is made, and each task runs once on the tier it was assigned. The retry strategy optimizer prices a cheap-first policy where every task starts on the cheap tier and falls back to the premium tier after a detected failure. Delays, backoff and jitter belong to the rate-limit backoff planner; neither this page nor the retry optimizer computes them.

Can the quality gap be negative?

Yes. When the premium tier's pass rate on easy work is lower than the cheap tier's, the routed mix is both cheaper and more accurate than the all-premium baseline at those inputs. Our routing study measured the premium tier refusing 75 to 86% of benign benchmark tasks, collapsing the raw pass rate to 14 to 18% once refusals count as failures. At those rates the refusal mechanism alone can push the premium tier below the cheap one on easy work. A negative quality gap is a real outcome, and the page reports it as such rather than clamping it to zero.

How do I know if the quality gap matters?

Test whether the gap sits inside your tolerance on the acceptable-error-rate calculator. If you are comparing two tier-level pass rates on paired items, McNemar's test tells you whether the difference is real. Turn the extra failures into a dollar figure on the expected-cost-of-failure calculator.

What does the break-even misroute-down rate tell me?

It is the value of the misroute-down rate at which the saving per correct answer reaches zero. When the crossing exists, the page states whether the saving sits below or above the crossing. In the standard case, routing saves below the crossing and the extra failures erase the saving above it. When the relationship reverses, the saving grows above the crossing and is gone below it. The page suppresses the figure and states the reason when no crossing exists inside zero to one.

Where does the policy side of this live?

Which task belongs on which tier is a policy question rather than an arithmetic one, and the guide to routing Claude models by task answers it. This page takes the split as given and prices it.