LatentEval
Reliability testing

Why AI agents hang: timeouts, stalls, and stops that don't

Agent runs hang in three shapes: an unbounded wait, an inactivity timeout a slow stream keeps alive, and a cancellation the work declines. Verified framework defaults, and how to tell them apart.

Part of Agentic AI testing beyond a single eval run

Reliability testing

In brief

5 POINTS
  • A hang has no bound, a stall has a bound counting the wrong quantity, and an ignored stop is a bound with no authority.
  • The Python HTTP clients under agent code count silence on the socket, so a response that dribbles slowly outlives the timeout you set.
  • SDK timeouts apply per attempt, so two automatic retries turn a ten-minute budget into thirty minutes of wall clock.
  • The MCP specification says receivers should stop on a cancellation notification and may ignore one they cannot honor.
  • Give the whole run one wall-clock deadline at the top and derive every layer's timeout from what remains.

Picture a run still in the process table. The last span in its trace opened and never closed, no exception fired, no retry counter moved, and the error rate on the dashboard is flat because nothing has failed yet. It may never fail.

Runs like that are also the ones that fall out of the record, and Failure as a Process lost them in two separate cuts. That July 2026 preprint from University College London and Nanjing University has not been peer reviewed. Its design called for 5,040 trajectories: 240 Terminal-Bench tasks run under 21 model and scaffold combinations. It collected 3,843, because the other 1,197 runs terminated abnormally or failed to complete. Keeping only the 89 tasks that every combination finished then took the annotated set to 1,794. On the first of those cuts the threats-to-validity section is candid: “Dropping timeout and incomplete trajectories may introduce bias, but such trajectories are outside our study scope.” That is why no figure for how often agents hang appears on this page. The trajectory study nearest to the question dropped the runs that would answer it before it started counting.

The same exclusion happens quietly inside your own numbers. A run that never returns is a run you cannot score, so it falls out of the denominator of the next pass rate you publish, and the rate goes up. Our within-Anthropic reliability benchmark prints what share of the index each model actually covered, because refusals and cap-censoring remove requests the same way, and its reporting protocol puts timeouts and rate limits in those same columns. Answer coverage is the name for that share.

Separate the three failures before you tune anything. A hang is a wait with no bound at all, a stall is a bound that counts the wrong quantity, and an ignored stop is a bound that fired into code with no authority to end the work. Raise a timeout you have not classified and the symptom moves to a different layer while the run stays exactly as stuck.

Three failures wear the same face in a trace

A hang is a wait that nothing in the code path will end.

The only things that can end it are outside the program: an operator, a container eviction, a CI runner’s own kill. To identify one, ask whether a timer exists at all on the specific wait that is open right now.

A stall has a timer, and the timer never fires because it counts something other than elapsed time. Both of the Python HTTP clients below count silence on the socket, and the clock resets whenever a byte arrives, so a response that trickles on forever keeps its own guard alive.

An ignored stop begins with a bound doing its job. A timeout fired, or a user canceled, or an orchestrator sent a cancellation notification, and the work carried on regardless. Every bound here looks correct when you inspect it, because every bound is correct; the signal simply arrived somewhere that was permitted to decline it. To identify one, watch the resource after the signal lands: the process, the socket, the open file handles.

Three timelines compared: a hang's wait has no timer at all and never returns; a stall's inactivity clock is reset by every arriving byte and never reaches its limit; an ignored stop's cancel is sent, the caller returns, and the work carries on.
The same stuck trace hides three different anatomies. A hang has no timer on its open wait. A stall's inactivity timer is reset by every arriving byte, so it never fires. An ignored stop is delivered, but the signal lands where it is free to be declined: the caller returns while the work carries on. Which anatomy you have decides which layer owns the fix. Structural diagram of the three failure shapes. No measured data.

Start at the wire. The timeout nearest the bytes reads like a duration limit, and it is measuring something else.

A read timeout guards silence, and a slow stream is never silent

Python’s requests is the clearest case, and its own quickstart guide says so without hedging: “Nearly all production code should use this parameter in nearly all requests. Failure to do so can cause your program to hang indefinitely.” The same page defines what the parameter actually bounds. The exception fires “if the server has not issued a response for timeout seconds (more precisely, if no bytes have been received on the underlying socket for timeout seconds).” So the guard tracks silence on the wire and says nothing about how long the whole download takes. Pass no timeout at all, and requests do not time out.

httpx is safer by default and works the same way underneath. Its documented behavior is to “raise a TimeoutException after 5 seconds of network inactivity.” Five seconds of silence is a tight guard for a REST call and no guard at all for a streamed completion, which is engineered to break silence constantly.

Work an example with numbers picked purely for the illustration. A streaming call sits behind a 60-second read timeout, and the model emits one token every 4 seconds because the provider is saturated. Every token resets the clock, so the guard never trips. At 2,000 tokens the single call occupies just over two hours of wall time, and the agent loop above it waits with a healthy connection and a live cursor throughout. Change any of those inputs and the conclusion holds: an inactivity timeout bounds silence, so it cannot bound duration.

Retries multiply the budget you thought you set

SDK timeouts are per attempt, and the retry policy sits above them. The OpenAI Python SDK states that “By default requests time out after 10 minutes” and that “Certain errors are automatically retried 2 times by default, with a short exponential backoff.” The Anthropic Python SDK ships the same shape in code: DEFAULT_TIMEOUT = httpx.Timeout(timeout=10 * 60, connect=5.0) alongside DEFAULT_MAX_RETRIES = 2.

Compose those two documented defaults and one call site can occupy three attempts of ten minutes each, so about thirty minutes of wall clock before any backoff between attempts is added, from a line of code whose author believed they had set a ten-minute ceiling. That figure is arithmetic on published defaults rather than something we observed. Both SDKs enforce that ceiling per attempt, and the call signature doesn’t say so.

Timeline in minutes: attempt one runs to its ten minute timeout at a dashed line marking the ceiling the developer set, then two automatic retries each run ten more minutes, reaching about thirty minutes of wall clock.
Each SDK timeout bounds one attempt, and the retry policy sits above it. With the documented ten-minute default and two automatic retries, the ceiling a developer set at ten minutes can hold the call site for about thirty. Arithmetic on documented SDK defaults (OpenAI and Anthropic Python SDKs, read 2026-08-02: 600 s per-attempt timeout, 2 automatic retries). Worst case shown; backoff gaps not to scale. Not a measured trace.

Retries interact badly with the layer above, too. A client that abandons a request the server is still computing, then immediately issues another, pays twice for one answer. That is the seed of a correlated retry cascade. The standing mitigation is to let a saturated stage signal back that it cannot keep up instead of absorbing the pressure silently.

What each layer actually bounds

Every default below was read from the source or the vendor’s own documentation on 2 August 2026. Defaults move, so re-check the row rather than trusting the table.

TABLEShow full table (8 rows)Showing full table (8 rows)
LayerDefault boundWhat the clock countsWhat it leaves unbounded
requests (Python)nonenothingeverything; the call waits as long as the socket stays open
httpx (Python)5 snetwork inactivitytotal duration of a slow but chatty response
OpenAI Python SDK600 s per attempt, 2 auto-retriesone HTTP attemptwall clock across the retry sequence
Anthropic Python SDK600 s per attempt, 5 s connect, 2 retriesone HTTP attemptthe same retry multiplication
MCP TypeScript SDK60 s per request; progress does not reset it; no maxTotalTimeoutone request round tripserver-side work that continues after the client gives up
LangGraph1000 super-steps since v1.0.6graph stepsseconds; one blocking step is invisible to the counter
CrewAImax_iter 20, max_execution_time with no stated defaultagent iterationswall clock, unless you set the execution time yourself
OpenAI Agents SDKmax_turns 10 in the SDK source, raising MaxTurnsExceeded; None disables itloop turnsthe duration of any single turn

Every bound in the third column counts one unit at one layer: a stretch of silence, a single HTTP attempt, a graph step, a loop turn. One row reaches the quantity a task owner is watching. CrewAI’s max_execution_time is an elapsed-wall-clock bound on the agent, and it ships with no stated default, so until you set it yourself nothing in this stack bounds how long the whole task runs.

A cancel is a request the other side may decline

The Model Context Protocol writes the ambiguity down in normative language. Its cancellation utility says receivers of a notifications/cancelled message SHOULD stop processing and free resources. It then lists when they MAY ignore one: the request is unknown, processing already completed, or “The request cannot be canceled.” The sender, meanwhile, “SHOULD ignore any response to the request that arrives afterward.” Both sides are told to handle the race gracefully, which is a specification conceding that a cancellation is advisory.

Language runtimes make the same concession. The Python documentation is blunt that Task.cancel() “does not guarantee that the Task will be canceled,” because the coroutine gets a chance “to clean up or even deny the request by suppressing the exception.” Suppression is discouraged and rare. The sharper problem sits in asyncio.wait_for, the helper most timeout code reaches for first: it cancels the task, then waits for that cancellation to land before it returns.

try:
    await asyncio.wait_for(run_agent_step(), timeout=30)
except TimeoutError:  # built-in TimeoutError on Python 3.11+; asyncio.TimeoutError before it
    # wait_for cancelled the step, then waited for that cancellation to land.
    # Per the asyncio docs: "the total wait time may exceed the timeout".
    log.warning("step exceeded its budget")  # the step's finally block may still be running

So a stop signal only ends work when something on the other end holds a handle with authority over it: a process to kill, a socket to close, a cancellation token the worker actually polls between units of work. Anything less is a polite ask traveling over a channel that documents its own right to be ignored. Once you find the ask being declined, the next decision is what a partially-completed run should hand back. That is the job of a degraded but correct response, backed by a breaker that stops sending calls into the failing dependency.

A step cap answers whether the loop is cycling, and nothing about when it ends

Agent frameworks bound iteration, and they say so plainly. LangGraph’s graph API documentation sets a recursion limit on super-steps and raises GraphRecursionError when it is hit, with the default at 1000 steps from version 1.0.6. CrewAI documents max_iter with a default of 20 and a max_execution_time that carries no stated default. The OpenAI Agents SDK raises MaxTurnsExceeded when the turn budget runs out, at a default of 10 turns set in its own run_config.py, and max_turns=None removes the ceiling entirely.

Every one of those bounds is useful, and every one of them answers the same narrow question: is this loop going round forever? CrewAI’s max_execution_time is the only one in this group that reaches duration, and it arrives switched off. The bound that does reach duration is a single wall-clock deadline held at the top of the run, with every layer below it deriving its own limit from what remains and the total read off the latency the stack actually produces rather than picked as a round number. A raised step cap also trades one failure for another, since a graph that cycles now runs 1000 super-steps of tokens and minutes before it errors.

Pull a cap in the other direction and you buy the opposite defect. A run that stops before the work is done and reports success is premature termination, which is the mirror image of everything on this page and has its own detection problem, because the run looks finished when it isn’t. Both failures are worth measuring off the same suite, and moving one number while the other goes unwatched is how a step cap gets tuned into a different bug.

Telling the three apart while the run is still stuck

Three checks, in this order, and stop at the first one that answers.

Watch for arrival first. Tail the socket, the stream, or the span emitter and see whether anything at all is landing. Bytes arriving means a stall. Silence means a hang or an ignored stop.

Then look for the timer. Find the exact call that is open and confirm a timeout reached it, since a value sitting in a config file proves nothing about the call site. A missing bound is a hang, and the fix there is a bound rather than a larger one.

Last, send the stop and watch the resource rather than the client. If the caller returns promptly while the CPU stays busy, the connection stays established, or the child process keeps its file handles, the cancellation was declined and you have an authority problem at that boundary.

Instrumenting for this afterwards is cheap. Emit one heartbeat per step carrying a monotonic timestamp, and record the deadline each step was given next to the time it actually consumed. Two numbers per step make all three failures legible in a log, and they give failure attribution something to work backwards from.

Finding out before the run is stuck means manufacturing each of the three shapes on purpose: a tool call that never returns, one that trickles bytes forever, and one that declines the cancel. The flagship profiler is designed to inject those three returns at a chosen boundary and report how often a run notices each of them against how often it simply keeps waiting. It has not shipped, so the notice rate for your own boundaries is a measurement you run rather than a figure you read off this page. Deliberate fault injection at the same boundary is how you run it, and the measured runs we publish show the form the answer should arrive in: the count behind every rate, with an interval around it.

Set one deadline above the stack and let every layer inherit it

Give the run a wall-clock deadline at the top, computed once when the request arrives, and pass the remaining budget downward so each layer derives its timeout instead of guessing one.

Four practices keep that deadline honest once it exists. Set an explicit timeout on every HTTP client you construct, including the ones inside libraries you did not write. For anything streamed, guard the token gap and the total duration separately, because the first catches a dead connection and the second catches a live one going nowhere. Log the deadline beside the elapsed time on every step. Count abandoned runs and publish that count next to your pass rate, since a rate computed only on the runs that came back is a rate with a selection problem baked into it. Recomputing it over the full denominator, and putting an interval on the rate that results, is what keeps that rate comparable from one run to the next.

None of that earns its cost if every run in your system is one request with an explicit client timeout on it. You already hold the bound, and threading a budget through a single layer buys nothing.

Two neighboring symptoms produce traces that look close enough to confuse a first read. A tool return chopped at a size limit leaves the loop spinning on incomplete data, covered in tool output truncation; a run that finishes and lies about the outcome is silent failure. Rule both out before you conclude a run is hung, and read the trajectory evidence on where coding-agent failures start if you want the wider anatomy. The method for testing any of this past a single green run sits in agentic testing beyond one eval pass, and the budgeting view of retries, fallbacks and human gates lives in making agents reliable in production.

The rule we ended up with is to treat a missing deadline as a defect of the same class as a missing error handler, and to catch it at the same moment in review.

Sources

  1. Failure as a Process: An Anatomy of CLI Coding Agent Trajectories Published
  2. Failure as a Process, threats to validity (preprint PDF, v1) Published
  3. Requests: Quickstart Retrieved
  4. HTTPX Advanced Usage: Timeouts Retrieved
  5. openai-python README Retrieved
  6. anthropic-sdk-python, src/anthropic/_constants.py Retrieved
  7. modelcontextprotocol/typescript-sdk, shared/protocol.ts Retrieved
  8. Model Context Protocol specification, Cancellation Retrieved
  9. Python documentation, asyncio Coroutines and Tasks Retrieved
  10. Python documentation, asyncio.wait_for Retrieved
  11. LangChain documentation, LangGraph Graph API Retrieved
  12. CrewAI documentation, Agents Retrieved
  13. OpenAI Agents SDK documentation, Running agents Retrieved
  14. openai-agents-python, src/agents/run_config.py Retrieved
  15. Claude Fable 5 vs Opus 5 vs Opus 4.8 reliability benchmark Retrieved