Independent (naive)
35.8%
INSTRUMENT | Propagation & containment
Multi-Step Agent Reliability with Shared Failures (Cascade Simulator)
5 cited sources
Price a multi-step agent chain when its steps share a cause: end-to-end reliability under common-cause failure beside the independent figure, plus the retry ceiling that shared failures impose.
A common-cause failure is one event that fails several steps at once: a degraded provider, poisoned shared state. Twenty steps at 0.95 succeed 35.8% of the time if independent and 47.8% when 30% of failures trace to a shared cause, so correlation raises the all-steps-clean rate while enlarging any single bad event.
The cost is the retry ceiling, 98.5% here, which no in-run retry passes, since it meets the same cause. For the independent baseline alone, use the independent-steps calculator. Built on how one fault spreads through an agent topology.
Showing your last valid result. Update the inputs above to recompute.
47.8%
Dependency-aware end-to-end over 20 steps at 0.95 per step with a common-cause fraction of 0.3; independent steps give 0.95^20 = 35.8%.
Gap, dependency-aware minus independent
+11.9 pts
The same chain priced two ways: independent steps against the shared-shock model.
| Metric | Independent | Dependency-aware |
|---|---|---|
| End-to-end success | 35.8% | 47.8% |
| End-to-end failure | 64.2% | 52.2% |
| A shared cause fails the run, per run | 0.0% | 1.5% |
| Failed runs that end at a shared cause | 0.0% | 2.9% |
| Pairwise failure correlation | 0.00 | 0.29 |
| Retry ceiling for in-run retries | 100.0% | 98.5% |
| One retry per step | 95.1% | 96.0% |
| Two retries per step | 99.8% | 98.4% |
Dependency-aware end-to-end success as the common-cause fraction moves, which is usually unknown; your value is marked.
| Fraction (β) | Success | Gap vs independent | Failure correlation | One retry per step |
|---|---|---|---|---|
| 0 | 35.8% | 0.0 pts | 0.00 | 95.1% |
| 0.1 | 39.4% | +3.6 pts | 0.10 | 95.5% |
| 0.2 | 43.4% | +7.5 pts | 0.19 | 95.8% |
| 0.3 (entered) | 47.8% | +11.9 pts | 0.29 | 96.0% |
| 0.5 | 58.0% | +22.1 pts | 0.49 | 96.2% |
| 0.7 | 70.5% | +34.7 pts | 0.69 | 96.0% |
| 0.9 | 86.0% | +50.1 pts | 0.90 | 95.4% |
| 1 | 95.0% | +59.2 pts | 1.00 | 95.0% |
Each step's failure probability split into the part a shared cause explains and the part the step owns alone; the two add up to the step's failure rate.
| Step | Per-step | Shared part | Own part | Note |
|---|
The per-step reliability budget for your target, under both models.
| Metric | Independent | Dependency-aware |
|---|
The per-step budget as the common-cause fraction moves; your value is marked.
| Fraction (β) | Required per-step |
|---|
dependency-aware = (1 - s)(1 - βq_max) · Π [p_i / (1 - βq_i)]; independent = Π p_i; uniform: p^N / (1 - βq)^(N-1)How?
How this is calculated
The multi-step agent reliability calculator treats a chain as N steps that fail independently, so twenty steps at 0.95 succeed end to end 0.95^20 = 35.8% of the time. This simulator keeps the same per-step rates and adds one dependence structure: a beta-factor common-cause model in the probabilistic risk assessment tradition, extended so that steps with different reliabilities are hit in order of weakness. Each step's failure probability qi = 1 - pi splits into a shared part, β · qi, and an independent part. One shared draw per run drives every step's shared part, so a shared cause severe enough to fail a reliable step also fails every less reliable step, and the run survives the shared cause only when that draw clears β · qmax, the weakest step's shared part. The independent part of each step is then set so its overall failure rate stays exactly the qi you entered: given the shared cause spared it, the step succeeds with probability pi / (1 - β · qi).
Worked example, the preloaded inputs. At 0.95 per step with β = 0.3, each step's 5% failure splits into 1.5 points a shared cause explains and 3.5 points the step owns alone. The step's own residual is slightly higher, 3.55 points (1 - 0.95/0.985), because it only gets to fire on the 98.5% of runs the shared cause left alone; 1.5% plus 98.5% of 3.55% is the 5% you entered. The shared cause fails 1.5% of runs. Given it did not, each step succeeds 0.95/0.985 = 0.9645, and twenty of them 0.9645^20 = 0.485; times 0.985 gives 0.478. The uniform closed form says the same thing at once: p^N / (1 - βq)^(N-1) = 0.358 / 0.985^19 = 0.478. The independent product charged the 1.5% once per step, twenty times over; this charges it once, and that is the whole 11.9-point gap. At β = 0, and with no shared cause outside your rates, the two agree exactly, so entering 0 reproduces the sibling calculator's numbers. At β = 1 the chain fails as one unit and the answer is the weakest step's rate, 0.95, which is the sibling's weakest link. Everything in between is what this tool prices.
Which way the independent figure errs. With the per-step rates held fixed, positive dependence raises the all-steps-clean rate, so the independent product is pessimistic about it; the site's own analyses of production agent reliability and multi-agent failure modes state the same result. What dependence worsens is everything else about a failure. Failures cluster into the same runs and, in production, into the same hours, so a shared cause is an incident rather than a flake, and every concurrent run that shares it fails together. And the per-step rates may not contain the shared cause at all: rates measured on clean, isolated evals on a healthy day never saw the provider degrade, and then the independent product is optimistic. The optional shared cause outside your per-step rates prices that case. It multiplies the dependency-aware figure by 1 - s and leaves the independent product where it was, because the independent product never knew; with it in play, your entered rates do not contain that cause at all, and no product of them can.
Retries under a shared cause. A retry inside the run re-rolls only the independent part of a step's failure: the same shared cause is still there when the step runs again. One retry per step drops step i's independent part from ri to ri^2 and leaves its shared part alone, so in-run retries can never beat the retry ceiling (1 - s)(1 - β · qmax), 98.5% in the example. The independent model's lever, 1 - (1 - p)^2 per step, has no ceiling, and its two-retry claim of 99.8% clears both the 98.4% two retries actually reach here and the 98.5% ceiling itself. Which delivery policy survives which failure, and at what cost, is priced in the retry strategy optimizer; a fallback to a path that does not share the cause is the lever that escapes the ceiling, and its value is exactly what this model cannot see.
The failure correlation. The pairwise failure correlation is the Pearson correlation between two steps' failure indicators under the same model: 0.29 in the example, against 0 under independence. For reliable steps it sits close to β, so a team with paired step outcomes can set β from a measured correlation. For steps with different rates the mean pairwise correlation cannot reach 1 even at β = 1, because two indicators with different rates cannot be perfectly correlated; the list mode reports the mean over all step pairs.
The sweep and the budget. β is rarely measured. Where the risk-assessment literature has no data it falls back to a generic screening value near 0.1 with roughly a factor-of-two spread (the NASA guide in the sources), which is why the sweep table walks β from 0 to 1 with your value marked: the closed form is monotone in β, so if β is bracketed, the answer is bracketed. Solve mode inverts the closed form to give the per-step budget a target demands under the shared cause, beside the independent budget target^(1/N). Positive dependence relaxes the budget, because the shared cause is charged once rather than N times; a target above 1 - s is unreachable at any per-step reliability.
Counts, the interval, and the simulation. Enter a step as eval counts (c of n) and the point estimate is c/n with a Wilson score interval at your chosen confidence; the per-step bounds are carried through both models, giving a band on the estimate rather than a joint confidence region. The simulation samples the identical generative model with a seeded generator, so the same inputs and seed reproduce the same counts; its rate carries its own Wilson interval and the closed form should sit inside it. It exists as an independent path to the same number and as a tally of what triggered each failed run; it adds no information the closed form lacks, and it is skipped when the run count times the step count, plus two draws per run for the shared causes, would exceed ten million.
Honest limits. β and s are modeling inputs, not measurements. The model has one shared cause with one severity ordering, so a step exposed to a cause the other steps do not share, or two unrelated shared causes, are approximated rather than modeled. It does not price cross-run incident clustering (a provider incident that spans an hour of runs), silent state corruption that fails a later step in a later run, or the cost of a failure once it happens. Why agent faults arrive in correlated clusters, and what contains them, is worked through in cascading failures in agent systems.
Formula: dependency-aware = (1 - s)(1 - βq_max) · Π [p_i / (1 - βq_i)]; independent = Π p_i; uniform: p^N / (1 - βq)^(N-1)
Questions
Why does correlation raise the end-to-end success rate instead of lowering it?
Because the per-step rates are held fixed. Each step still fails as often as you said; dependence only decides whether those failures land in the same runs or in different ones. Independent failures spread across many runs and spoil many of them. Correlated failures pile into fewer runs and leave more runs entirely clean, so the all-steps-clean rate goes up, toward the weakest step's rate at β = 1. What goes down is everything else: the failures that remain arrive together, take several steps at once, and often take several concurrent runs at once. The independent figure is pessimistic about the clean rate and silent about the clustering.
When is the independent figure optimistic instead?
When the shared cause is not inside your per-step rates. Rates measured on isolated evals on a healthy day contain no provider degradation, no bad upstream input, no shared-state corruption, so both the independent product and the dependency-aware figure built on those rates miss it. Enter the per-run probability of that unmeasured cause in the optional field; the dependency-aware figure drops by that share and the independent product does not move, which is the honest picture of what the independent product never knew. Rates taken from production traces already contain the shared cause, and then the common-cause fraction alone is the right input.
What counts as a shared cause in an agent chain?
Any single event that can fail more than one step. Four common ones: a model provider that degrades, so every step calling it degrades together; an upstream input that arrives malformed, so every step reading it is wrong; a retriever or index that hands bad passages to every step that queries it; and shared state poisoning, where one step writes a corrupted value into scratch memory or a shared store and every later step that reads it inherits the error. A rate limit, an expired credential, and a bad deploy of a shared prompt template behave the same way. Failures whose cause is local to one step belong in the independent part, the 1 - β share of each step's failures.
How do I estimate the common-cause fraction?
From paired step outcomes, if you have them: the pairwise failure correlation the tool reports sits close to β for reliable steps, so match β to the correlation you measured between two steps' pass/fail records over the same runs. From incident logs, otherwise: the share of a step's failures that a shared trigger explains (a provider incident, a malformed upstream batch, a corrupted cache) is a direct estimate of β. Where nothing has been measured, the risk-assessment literature falls back to a screening value near 0.1 with roughly a factor-of-two spread; enter that, read the sweep table, and treat the range as the answer. How far the shared-cause patterns from service outages carry over to agent chains, which is what makes an incident log usable as evidence here, is tested in what microservice cascades teach about agent cascades.
Why can a retry not beat the retry ceiling?
A retry re-runs the step inside the same run, against the same shared cause. If the provider is degraded, the upstream input is malformed, or the shared state is corrupt, the retry meets the same condition and fails the same way; only the step's own independent failures are re-rolled. So retries drive the independent part toward zero and leave the shared part untouched, and the chain converges on (1 - s)(1 - βqmax), the retry ceiling: 98.5% on the loaded inputs, which no number of in-run retries passes. Escaping it takes a lever that does not share the cause: a fallback path on a different provider, a re-fetch of the upstream input, a reset of the shared state. Those levers are priced against each other on a single step by the retry, verification and fallback optimizer.
How large is the blast radius when one shared cause fires?
Every step exposed to the cause, in every run that meets it. That is the blast radius, and the model prices how often it fires: at 0.95 per step over twenty steps with a common-cause fraction of 0.3, the shared draw alone fails 1.5% of runs. It stops at the run boundary, though. In production those runs cluster in time, so one event often takes concurrent runs together, and no per-run figure shows that. Cascade containment is the lever that shrinks the radius: a bulkhead per failure domain so a degraded dependency cannot reach the rest, a circuit breaker on the shared call, and a checkpoint so a hit run resumes instead of restarting. None of those are priced here; this model prices the chain as it stands.
Should I use this or the independent-steps calculator?
Start with the independent-failure baseline for the per-step budget and the weakest link; its numbers are this tool's β = 0 column when the extra shared cause is left blank. Come here when the steps share a dependency (a model provider, a retriever, an upstream service, a store) and you want to know how far the independent figure can be off, what a retry can still buy, and what budget the target really demands. The two agree exactly at β = 0 with no extra shared cause, and this tool converges on the sibling's weakest link at β = 1, so the sibling gives you the two ends of the range and this tool gives you everything between them.
Sources
- Series model (system reliability as the product of the component reliabilities: the independent baseline)
- Probabilistic Risk Assessment Procedures Guide for NASA Managers and Practitioners, chapter 7: modeling and quantification of common cause failures (the beta-factor model and its generic screening value)
- A reliability model for common mode failures in redundant safety systems (the original beta-factor model)
- A Multivariate Exponential Distribution (the shared-shock model of dependent failure)
- Probable Inference, the Law of Succession, and Statistical Inference (Wilson score interval)