Fallback chain (model routing)

A fallback chain is the ordered sequence of models a single request walks when the model it named declines to serve it: the primary first, then each alternate in the order the caller listed them, until one returns an answer or the list runs out. Every hop carries the same prompt, the same tools and the same conversation, so the request arriving at entry two is identical to the one entry one declined, and only the author of the reply changes.

The caller does not always write that list. Since July 2026 the Anthropic parameter has also taken a default mode, where the provider picks the substitute itself and routes by the category of the refusal rather than by an order anyone declared, and it arrives under a beta header of its own. That leaves the served-by record as the only account of what the chain reached for, since a caller who named no candidates cannot infer the answer from a config file.

Routing is the wider of the two decisions. A model router picks one model per request before the request is sent, weighing cost, latency and the shape of the task; the chain governs only what happens once that choice comes back empty. On Anthropic’s API the trigger narrows again, because only a safety-classifier decline moves a request to the next entry, while a rate limit, an overload or a server error is returned to the caller as it stands (Anthropic, “Refusals and fallback”, as of 2026-08).

A chain configured against refusals sits idle through an outage.

Our pre-registered routing study measured what a live chain does to the numbers underneath it. Running a two-entry chain from Claude Fable 5 to Claude Opus 4.8 on short machine-checkable tasks at low effort, the fallback fired on 20 of 28 calls, none stayed refused, and the pass rate with rescue on reached 96.4% (27 of 28, 95% CI 82.3–99.4%). Opus produced every one of those 20 answers while the caller paid the premium tier’s rate for them. Read only the response text and the run looks like 28 clean completions from the model that was asked.

That is the tension a chain carries into every rate computed downstream of it. It lifts answer coverage by converting refusals into completions, and it makes the identity of the answering model a fact you have to record rather than assume. A refusal rate published from a system with a chain in front of it describes the last model that ran rather than the one named in the request. Log the served model on every call before you publish anything computed on those answers.

How to measure a fallback chain

A chain is a mechanism rather than a single quantity, so three proportions over one denominator carry the claim, and the denominator is requests entering the chain. Per-entry fire rate gives the share that reached each alternate, which tells you whether entry two is doing the work or entry one is, and terminal-refusal share counts the requests that walked the whole list and came back declined anyway. The third, and the one most systems cannot produce until they add the field, is the served-by distribution: every delivered answer attributed to the model that actually wrote it.

Each of the three is a proportion, so each carries an interval rather than a bare percentage, and the pass-rate confidence interval calculator takes a k-of-n count straight to a Wilson or Clopper-Pearson bound. Instrument at the call site, because the response is the only place that record exists. response.model names the model that produced the returned message and usage.iterations holds one entry per attempt, and since a fully declined chain still writes a fallback_message entry for its last hop, read stop_reason before crediting any rescue. Those field names are scoped to one surface: server-side fallback is a beta on the Claude API as of 2026-08, gated behind a dated beta header, absent from the Message Batches API, and absent from the Bedrock, Google Cloud and Microsoft Foundry deployments, where the same job falls to a client-side chain you write. The two fields to log on every call are the whole instrumentation.

One property of the mechanism will distort all three rates if you do not know about it. Anthropic skips the fallback attempt when the fallback model is itself rate limited or overloaded, returning the preceding refusal instead, and declined attempts still count against their own model’s rate limits. So the fire rate falls at the moment traffic is heaviest, which is also the moment a healthy-looking report is most likely to be read as evidence the chain works. Size the alternates for the refusal volume you expect.

Fallback chain vs retry

A retry re-issues the same request to the same model, betting that sampling non-determinism lands somewhere better the second time. A chain changes the responder instead, on the bet that different weights behind a different safety classifier will treat the same prompt differently. The two of them answer separate classes of failure.

Retry wins where the fault is transient and the model is fine: an overload, a dropped connection, a 500 with nothing to do with the prompt. On Anthropic’s API a chain will not fire for any of those, so a stack carrying a chain and no retry has no cover for the most ordinary outage there is. Where the fault reproduces, only the chain helps, because a policy decline on a benign prompt or a capability the model does not have comes back identical on the second attempt. That is why microservices resilience patterns port unevenly, with naive retry on a semantic fault porting worst of all, and why correlated retries build a retry storm that a chain would have spread across two models.

Count retries and chain hops on separate counters. A single “recovered” figure that folds them together hides which of the two mechanisms is carrying your availability, and the two fail on different days.

Whether the chain fired and whether the answer degraded are two different counts

Graceful degradation is a property of the answer a caller received: reduced in scope, freshness or confidence, still correct, and legible as reduced. A fallback chain is configuration that runs whether or not anything useful comes out of it. The chain is the mechanism; degradation is a verdict on the result.

Firing and degrading are separate events, and the reporting consequence differs depending on which one happened. A chain can fire on every request and produce no degradation at all, when the second model answers as well as the first would have. That is where our own routing run landed: the rescued answers were correct, the delivered pass rate went up, and what changed underneath was the price and the identity of the responder, with no declaration of either in the payload. A chain can also fire and reduce the answer quietly, giving up scope or freshness while the reply still reads as complete. Degradation counts as graceful only when the caller can tell, and a chain declares nothing on its own. Track the chain’s hop counts for what was tried, and the degraded-result split for what the caller got.

Fallback chain vs a CSS font stack

The older and far larger meaning of an ordered fallback list belongs to typography. MDN describes font-family as “a prioritized list of one or more font family names”, where “the browser will select the first font in the list that is installed or that can be downloaded using a @font-face at-rule” (MDN Web Docs, as of 2026-08). DNS resolver ordering and telephony failover lists run the same discipline, the ordering logic really is shared, and borrowing the phrase costs nothing.

Two things break the analogy once the list holds models. A font substitution is visible on sight, since letterforms and metrics change on the page; a model substitution returns a paragraph of the same length, register and confidence as the one the primary would have written, and the substitution shows up only in the response metadata. MDN’s standing advice, that a generic family belongs last because no font is guaranteed available, has no equivalent here either. Anthropic’s list caps at three entries, each distinct from the others and from the requested model, and a chain where every entry declines returns the last model’s refusal. There’s no serif of last resort to catch that, so the terminal case is the caller’s to handle.

One discipline survives the analogy: report the chain as three rates with intervals rather than describing it as configured. Saying “we have a fallback chain” describes a config file; per-entry fire rate, terminal-refusal share and served-by distribution describe what happened to your traffic.

The answer text carries no mark of which model wrote it, so the served-model log is the only record that makes a routing decision auditable afterwards. What reaching up the tier ladder actually buys runs through the routing question in full, and the case for treating a chain as one budgeted lever among retry, checkpoint and human-gate sits in the production reliability playbook. The three rates all come out of two logged fields.