Timeout budget (agent runs)

A timeout budget is a single wall-clock deadline held at the top of an agent run, from which every layer underneath derives its own limit by subtracting what the run has already spent. The derivation is what makes it a budget. A per-call timeout is chosen independently at each layer and bounds only the call it sits on, so a run can honor every timeout in the stack and still take hours, because nothing anywhere is tracking the total. A budget bounds the run, and each layer’s limit is whatever remains of it at the moment that layer is entered.

The difference shows up first in the retry path, where per-attempt bounds compose in a direction most authors do not expect. SDK timeouts apply to one attempt, and the retry policy sits above them, so two automatic retries turn a ten-minute ceiling into roughly thirty minutes of wall clock from a call site whose author believed the limit was ten. Our account of why agent runs fail to terminate works that arithmetic through against the published SDK defaults and carries the sources for each. A budget removes the multiplication by construction: the second attempt inherits what the first one left, and the third inherits what is left after that, so the sequence cannot outrun the deadline no matter how the retry policy is configured.

The second difference is the one that catches teams who believe they already have a bound. Agent frameworks do ship default ceilings, and every one of the common ones counts steps rather than seconds. The same investigation records LangGraph raising GraphRecursionError at a default recursion limit of 1000 super-steps from version 1.0.6, CrewAI documenting max_iter at 20 alongside a max_execution_time that carries no stated default, and the OpenAI Agents SDK raising MaxTurnsExceeded at a default of 10 turns, with max_turns=None removing the ceiling entirely. A step counter answers whether the loop is cycling. It says nothing about duration, because one blocking step is invisible to it, and a graph that never repeats a node can sit inside a single step for as long as the call underneath it stays open. A step budget and a time budget are different instruments, and a system holding only the former can hang indefinitely without ever tripping a limit.

How to derive a budget from measured latency

The usual way a deadline gets chosen is that somebody picks a round number. The better way is to read it off the latency distribution the system actually produces, which requires knowing the tail rather than the average, because the tail is what a deadline interacts with.

Our three-model reliability run has the shape of that problem on record. Across 232 completed synchronous calls, Kimi K3 returned a median reply in 20.7 seconds, a figure that would make a 60-second timeout look generous. Its 90th percentile was 431 seconds, and its slowest completed call took 2,138.9 seconds, just under 36 minutes. A further eight items never returned at all, the connection dying server-side mid-stream, and those are excluded from the 232 rather than counted as slow. The full distribution is in the three-model benchmark this figure comes from. The spread between the median and the 90th percentile is a factor of about twenty on one model, which is the argument against picking a round number: a deadline set from the median cuts off a large share of work that would have succeeded, and one set from the maximum sits so far out that everything the budget existed to catch passes underneath it.

So take the percentile as the input and decide the percentile deliberately. Measure per-call latency at each layer the budget will cover, on the models and tools you actually run, and set each layer’s expected consumption from a stated percentile of its own distribution. Sum those, add the retries you intend to allow, and the total is the budget the top of the run should hold. Publish the percentile you chose alongside the number, because a budget derived at the 90th percentile and one derived at the 99th are different products: the first trades completed work for predictable latency, the second does the reverse, and the choice belongs to whoever owns the user-facing promise.

Two conditions keep the derivation honest. The distribution has to come from your own traffic, since latency depends on prompt length, tool mix, and how saturated the provider is, and a vendor’s published figures describe neither your prompts nor your load. And the calls that never returned have to be counted somewhere, because dropping them silently makes the tail look shorter than it is. Eight dead streams excluded from a denominator of 232 is a manageable exclusion when it is stated and a distortion when it is not, the same accounting problem answer coverage exists to expose.

A deadline only ends work when something holds a handle with authority

Deriving the number is the easy half. Making it bite requires that some component can actually stop the work when the budget runs out, and in an agent stack that is frequently untrue.

The Model Context Protocol writes the ambiguity into its own normative language: receivers of a cancellation notification should stop processing and free resources, and the specification then lists the cases where they may ignore one, while telling the sender to ignore any response that arrives afterward. Both sides are instructed to handle the race gracefully, which concedes that a cancellation is advisory. Language runtimes make the same concession, since a canceled task gets a chance to clean up or to suppress the cancellation altogether. The upshot is that a stop signal ends work only where something on the other end holds a handle with real authority over it: a process to kill, a socket to close, a cancellation token the worker polls between units of work. Everything else is a request the far side is documented as being free to decline. The framework-level detail on all three, with the sources, sits in the hang, stall and ignored-stop diagnosis.

The practical consequence is that a budget should be enforced at the layer that owns a killable handle, and the layers above it should treat their own deadlines as reporting instruments rather than as guarantees. A budget enforced only in the orchestrator’s bookkeeping produces a run that is marked expired while the work underneath it continues to burn tokens.

Timeout budget vs premature termination

A budget that fires and an early stop are the same event seen from two sides, and the honest way to state the relationship is that the budget is a deliberate stop while premature termination is a failure, with the exit contract as the only thing separating them.

A cutoff that returns “incomplete, budget exhausted” together with partial state marked as partial is a bounded run behaving correctly, which is a degraded response that declares itself. The identical cutoff returning a completion claim is premature termination, because the run has handed back work that was never finished with nothing to say so. The stop is the same; the label depends entirely on what the run tells its caller. This makes the stop reason a first-class field rather than a log line, since an eval reading only outcomes cannot separate the two.

The two rates also move against each other. Tighten the budget and some runs that would have finished convert into early stops; loosen it to recover them and you pay in wall clock and tokens on runs that were never going to return. Both belong in one table computed off one suite, because a change that improves one while the other goes unmeasured has not been shown to help.

Where the pattern comes from

Timeout budgets are an import from distributed systems, where the idea is called deadline propagation: set one absolute deadline high in the stack and carry it down, so that the tree of calls emanating from an initial request all share the same deadline, and a downstream hop can see how much of the original allowance is left before it starts. Google’s SRE practice is the canonical statement of it, and gRPC implements the same shape in its wire protocol.

The port to agents holds well and costs something. Our audit of which microservices resilience patterns survive the move rates this one as holding moderately: the propagation mechanism transfers cleanly, and two things about agent work strain it. Agent steps have nondeterministic duration, so a hard cut is more likely to land mid-reasoning and leave partial state that a service call would not have produced. And there is a second budget services never had, tokens and dollars, which frequently binds before wall-clock time does. Both belong in the same budget object, drawn down by the same components, because a run bounded in seconds and unbounded in spend is only half bounded. When the constraint is that a downstream stage cannot keep up rather than that time has run out, the instrument is a signal that travels upstream and slows dispatch instead of a deadline.

A run with per-call timeouts at every layer and no total is a run nobody has bounded, and the fix is one number held at the top with everything below it doing subtraction. What each layer in a real stack actually bounds, and which of them leave duration open, is tabulated in the analysis of agent runs that will not end.