Bulkhead isolation (agent systems)

Bulkhead isolation partitions the resources an agent system shares, so that any one agent, tool or provider consumes only the slice assigned to it and a fault that would have drained the pool drains a single partition. Those resources are concrete in an agent stack: worker slots, the token budget held with each provider, the credentials a run can present to a tool, and the memory a branch reads from. The nearest neighbor is cutting a failing dependency off once errors cross a threshold, which watches health and then stops calls; a bulkhead reads only the pool, capping how much of it one caller holds at a time.

A bulkhead is a bound rather than a detector, so the figure describing one is a capacity, and a capacity means little without the saturation behavior beside it. Give the slot count, the queue depth behind it, and the fate of the request arriving when both are full. Tail latency decides whether the bound is worth its cost, and it is measurable per provider. Our three-model reliability campaign timed every synchronous call Kimi K3 completed: median 20.7 s, 90th percentile 431 s, slowest completed call 2,138.9 s, about 36 minutes (232 completed calls, with the eight items whose connections died server side left outside that denominator). Fable 5 and Sol ran mostly through batch queues in that campaign, so the tail describes one provider’s interactive behavior and holds only for that provider.

An unpartitioned pool will hand its last slot to a call that holds it for half an hour while still technically working.

The partition a bulkhead draws is what failure domain names, and containment rate scores that boundary once you inject a fault at it and count what crossed. Draw the partition early, because one agent’s output becoming another’s trusted input is only one of the two ways a fault spreads, and the second is a fault that starves work it never touched.

Sizing a bulkhead in an agent system

Start by naming what the pool actually hands out, since three pools usually wear one name: in-flight requests, the provider’s rate limit in requests and tokens per minute, and the process resources a waiting worker holds. Give each consumer class a cap on all three, sized against the longest a call can hold a slot rather than against the average. The average is set by the easy requests, and the pool is drained by the hard ones. Enforce a timeout of your own, since a partition sized against an unbounded wait bounds nothing.

Then state the saturation behavior, which usually goes unwritten. Reject immediately, queue to a bounded depth, or route the overflow into a reduced result that is still correct. Unbounded queueing recreates the exhaustion one level up, with the queue as the new shared resource, and backpressure is the signal that keeps it from getting there.

Report the pool size, the per-partition cap, the timeout you enforce, and the share of requests each partition rejected while saturated. That last figure is a proportion, so it carries a confidence interval like any other rate. The flagship profiler, once it ships, is designed to saturate one partition deliberately and report what fraction of runs in the neighboring partitions completed anyway, with a bootstrap interval on that fraction. Nothing on this page is that measurement; the rejected share is yours to count, and the pass-rate interval calculator will bracket it once you have the count and the denominator.

Instrument at the dispatch point where an orchestrator assigns work, inside each provider client, and at the MCP gateway if your tool calls funnel through one. Once every step reports a bounded failure rate instead of an open-ended hang, the system reliability calculator composes those per-step rates into an end-to-end budget.

Bulkhead isolation vs circuit breaking

A circuit breaker tracks the failure rate on calls to one dependency and opens once that rate crosses a threshold, so further calls fail instantly until a trial request finds the dependency healthy. Bulkhead isolation caps concurrent use of a resource and never revises the cap in response to anything. Either one can be carrying the whole load on its own. A dependency that answers slowly and correctly may never trip a breaker configured on errors alone, while the bulkhead still confines the waiting to one partition. A dependency that fails fast holds no resources worth capping, so the work falls to the breaker, which gives that service quiet and keeps correlated retries from amplifying the fault. Count a timeout as a breaker failure and the two cover each other, since a slow dependency is the case an error-counting breaker would otherwise hand to the bulkhead alone.

Bulkhead isolation vs failure domain

A failure domain is the set of agents, tools and shared state a single fault can reach before something stops it, read off a specific topology rather than stated globally. Bulkhead isolation is one of the mechanisms that decides where that set ends. A system with no bulkheads still has failure domains, discovered after an incident rather than chosen in advance, and in a hub-and-spoke fan-out the domain widens toward the full width of the workers. A system with bulkheads can leave its domain where it was: partition the worker pool cleanly, let every partition write to one memory store, and a corrupted note still reaches every branch. Read the domain to know what is exposed, install the bulkhead to change it, then measure how far a fault actually traveled and score the wiring with cascade resistance.

What the microservices bulkhead assumes about failure

The pattern is canonical outside this lane, and the canonical statement is worth reading first. Microsoft’s Azure Architecture Center documents it as isolating an application’s elements into pools so a fault in one does not cascade across the rest, names the ship’s-hull partitions behind the term, and points at connection pools, thread pools and semaphores as the things to divide. Its guidance now singles out AI and inference workloads as needing strict bulkheads, on account of deployment-level quotas and concurrency limits. Netflix’s Hystrix made the pattern operational for a generation of Java services through thread and semaphore isolation, and its README now records that the library is no longer in active development. An agent team inherits the idea and the vocabulary, and builds the implementation.

That construction assumes a fault is resource-shaped and that consumers fail independently. Agent workloads break the second assumption routinely, because a partition divides capacity while a poisoned document or an injected instruction rides the input and lands in every partition at once. Partitioning by consumer does nothing to a fault that travels with the work. The audit of which resilience patterns survive the port reaches that verdict pattern by pattern: bulkheads and breakers map across close to directly, while retry-with-jitter misleads, because a semantic fault reproduces on the retry.

So partition along the axis the fault travels. For exhaustion, partition concurrency, rate budget and timeouts per consumer class. For a corrupted artifact, partition state and credentials, so that what one partition writes another cannot read without a check in between, and a compromised tool session cannot present the union of every permission the run holds.

Sizing a partition is a topology decision before it is a capacity one, since the wiring fixes what a fault can reach and the pool fixes only how much it can hold. Cascading failures in agent systems follows one wrong output from its trigger to the boundary that stops it, and the survey of orchestration patterns and what each amplifies covers which wiring hands you which failure to partition against.