Exponential backoff with jitter

Exponential backoff with jitter is the retry-timing discipline in which each successive attempt waits longer than the one before it, and the length of that wait is drawn at random from the growing interval rather than taken as its exact value. The two halves answer two different problems. Exponential growth stops one client from hammering a dependency that is already struggling, giving it room to recover between attempts. The randomization addresses a problem the growth creates: clients that failed at the same moment and back off by identical amounts stay time-correlated, so they retry together, collide again, and reconverge on a longer and longer cycle. Jitter breaks that correlation by making two clients that failed together wait different amounts.

The pattern is settled practice in service engineering, and the canonical recommendation is unusually strong. Marc Brooker’s AWS analysis concludes that the return on implementation complexity of jittered backoff is large enough that it should be treated as a standard approach for remote clients.¹ That recommendation is sound in the setting it was written for, and the two limits below are where it stops carrying, both of which our own published work argues rather than inherits.

How to report whether your jitter is working

Jitter is a claim about the shape of a distribution, so a configuration value proves nothing about the result. What the mechanism promises is that attempts which used to arrive together now arrive spread out, and the measurement should read that directly.

Peak concurrent attempts at the dependency is the primary number, taken over a stated window and compared against the same window before jitter was introduced. A working implementation lowers the peak while leaving the total roughly where it was, which is the signature to look for: the same work, arriving less bunched. Where a fleet is large enough to sample, the inter-arrival distribution is the stronger evidence, since correlated retries show as a spike at the backoff interval and jittered ones fill in around it.

Report those beside the two counts a retry path always owes: attempts per originating request, over a stated window and a denominator of distinct originating requests, and the share of retries that eventually succeeded. The second is a proportion, so it needs an interval rather than a lone percentage, on the reasoning behind any eval confidence interval, and the pass-rate interval calculator brackets it from k of n. The full accounting of how those attempts multiply, and where to instrument for them, is in the entry on correlated retry cascades.

Jitter fixes correlation and leaves amplification untouched

The first limit is arithmetic. Spreading attempts across time changes when they arrive and leaves how many of them there are exactly as it found it, so a stack that generates 64 attempts from one user action generates 64 jittered attempts instead of 64 simultaneous ones. If the dependency is failing because it is over capacity in aggregate rather than over capacity in a single instant, jitter converts a spike into a plateau and the outage continues.

That multiplication comes from layering, since retry counts at successive layers compose as a product rather than a sum, and the standard remedy for it is a separate instrument: a retry budget capping how many retries a process may issue at all, after which requests fail outright instead of being re-sent. The worked arithmetic behind the product, and the SRE budget figures with their source, sit in the retry-storm entry’s amplification section. Backoff schedules and retry budgets are complementary controls, and a system carrying only the first has addressed the timing of its retry load without addressing its volume. Where the constraint is that a downstream stage cannot absorb the aggregate at all, the instrument is a signal that travels upstream and slows dispatch.

Jitter fixes transient faults and leaves semantic ones

The second limit is the one that matters most for agent systems, and it is about scope rather than mechanism. Every retry discipline assumes a fault that a later attempt can fix: the call dropped, the resource was briefly unavailable, the identical request succeeds once the dependency recovers. Backoff buys time for a recovery that is actually underway.

The dominant agent fault has a different shape. A call that succeeded and returned something confidently wrong is not waiting on anything, and re-issuing the same prompt against the same corrupted context reproduces the error verbatim, with a delay added. A malformed tool argument, a prompt that trips a guardrail, and a plan the model regenerates the same way each time all behave that way. Our audit of which resilience patterns survive the move from microservices names retry with jitter as the place the analogy leads you wrong, and the argument is about where the pattern applies rather than whether it works. Jitter genuinely helps an agent fleet hammering a rate-limited tool API, spreading retries so a set of workers does not synchronize into a thundering herd. Applied to a semantic fault it adds latency and cost to a result that was never going to change.

The practical consequence is to gate the retry on the fault type before the schedule ever runs. A transient fault on an idempotent step is close to free to retry; a semantic one needs a path that reaches a different result, which means a different model, a different prompt, or a verification gate that stops the wrong answer being read downstream at all. Our production playbook treats that fault-type check as the thing that decides whether retry belongs in the budget for a given step. Retrying harder against a metered provider is worse still, since a token bucket returns errors faster the more you push it and a sharp usage increase can trip an acceleration limit by itself.

Exponential backoff with jitter vs a circuit breaker

The two patterns act on the same traffic at different scales, and they are complements rather than substitutes. Backoff governs the timing of one client’s attempts against one failing dependency. A breaker governs whether attempts are sent at all: once a failure threshold trips, the breaker cuts calls to the dependency for a cooling period, so the load stops instead of arriving more politely spaced.

That difference decides which one to reach for. Where a dependency is briefly unavailable and recovering on its own, backoff alone is the lighter instrument and the breaker adds a failure mode. Where a dependency is down hard, backoff keeps sending doomed traffic on a lengthening schedule while a breaker stops it, and the breaker is what converts a slow bleed into a fast, visible failure the caller can route around.

Whether that conversion is a benefit is genuinely contested, and the sharpest statement of the doubt comes from the same author as the jitter recommendation, arguing that circuit breakers are designed to turn partial failures into complete ones.² The concern is that a threshold read off a noisy signal will trip on a dependency that was still serving most requests, taking down the remaining capacity in the name of protecting it. That argument is about breakers rather than about jitter, and it is worth carrying here because it sets the order of adoption: a backoff schedule is close to free and hard to misconfigure into an outage, while a breaker is a control with its own tuning surface and its own way of failing.

Two other distinctions are worth keeping straight. A thundering herd is a synchronization problem that one round of correlated arrivals is enough to cause, while a retry storm carries a feedback loop where each round of failures generates the next, and the entry that separates them sets out why the herd’s number is peak concurrency and the storm’s is attempts per request. Jitter is the shared cure for the timing half of both.

A jitter setting is easy to add and easy to believe in, and the check that it worked is a lowered peak at the dependency rather than a line in a config file. What the schedule bounds, what a budget bounds, and what neither of them reaches is worked through in the porting audit for microservices resilience patterns.