Agent turn hangs indefinitely when a streaming response stalls (no idle timeout in messagesApi/responsesApi SSE read loop)

[meganrogge]

Summary

claude-opus agent turns hang for the full harness wall-clock cap (eval: X_AGENT_STILL_RESPONDING, 0 output) when a single /v1/messages (Anthropic Messages API) streaming response stalls — the upstream/proxy returns HTTP 200 headers but never streams a terminating body and never closes the socket, and the client's SSE read loop in processResponseFromMessagesEndpoint has no idle timeout, so the turn never resolves or rejects. Found via terminalbench2 eval runs where it was the dominant error mode (~18 of 30 errored instances across 3 vscode-copilot-cli runs). The same unguarded read loop exists in the OpenAI Responses API path, so this is not Anthropic-specific in principle.

Where it hangs

extensions/copilot/src/platform/endpoint/node/messagesApi.ts → processResponseFromMessagesEndpoint:

for await (const chunk of response.body) {
    parser.feed(chunk);
}

No inactivity guard. When the body delivers 200 headers but no SSE bytes (and never ends), this loop blocks forever: no message_stop is parsed, the AsyncIterableObject never calls feed.emitOne(...) or feed.reject(...), and the ToolCallingLoop awaiting the turn hangs until an outer cap (the eval harness's 60-min limit; in product, effectively indefinitely / until user cancel) kills it.

The identical pattern is in extensions/copilot/src/platform/endpoint/node/responsesApi.ts → processResponseFromChatEndpoint (line ~907), so the OpenAI Responses path shares the vulnerability.

Evidence (from eval GitHub Copilot Chat.log)

A healthy turn logs [messagesAPI] message 0 returned. finish reason:[stop] → ccreq …| success | claude-opus… → (final turn) [ToolCallingLoop] Stop hook result: shouldContinue=false → clean exit.

A hung turn:

Instance	Signature	Evidence
circuit-fibsqrt	3 turns OK, 4th /v1/messages never returns	last ccreq …success…[stop] @ 20:50:54; capi-proxy sends req [10] @ 20:50:57 with no matching [messagesAPI] message returned / ccreq; silence → OTel flush 21:50:42 (60-min cap)
regex-chess	2 turns OK then hang	last return 21:08:27 → cap flush 22:08:14
train-fasttext	36 turns OK then hang	last return 22:30:41 → flush 22:36:46 (hung at min ~54)
winning-avg-corewars	60 turns OK then hang	last return 22:40:18 → flush 22:42:54; one completed turn took 133,191ms

The hang strikes an intermittent turn (2nd to 61st), confirming it is a per-turn streaming stall, not a prompt/content issue.

Affected eval runs (terminalbench2, vscode-copilot-cli)

Run	Model	Resolved / Failed / Err	timeout-hangs
27478622081	opus-4.6	48 / 28 / 13	9
27511500953	opus-4.6	52 / 27 / 10	6
27511511587	opus-4.7	60 / 23 / 5	3

Same instances PASS on the headless Copilot CLI control runs (different harness): 27479099364 (0 err), 27512067473 (1 err).

Why GPT-5.x was not hit

GPT-5.x routes through useResponsesApi → processResponseFromChatEndpoint (OpenAI Responses upstream), which is selected before the Messages path; only models advertising Messages in supported_endpoints (Anthropic) hit processResponseFromMessagesEndpoint. The streaming stall in these runs occurred on the Anthropic /v1/messages upstream, which OpenAI models don't use. This is routing luck, not structural immunity — the Responses path has the same unguarded read loop and would hang identically if its upstream stalled the same way.

Ruled out

• OTel consent modal — present on resolved instances too; a universal end-of-run screenshot artifact, not the cause.
• "Retry storm" / repeated x-request-id — that id is a per-session id, constant across all /v1/messages calls in a session; resolved instances repeat it 28–37× and pass.
• HMAC 401 near end-of-hang — a background model-metadata refetch firing ~55 min after the wedge; effect, not cause.

Proposed fix

1. Add a streaming idle timeout to both processResponseFromMessagesEndpoint (messagesApi.ts) and processResponseFromChatEndpoint (responsesApi.ts). Arm a watchdog that resets on each chunk; if no chunk arrives within ~120s, response.body.destroy(new Error('stream idle timeout')) so the for await throws and feed.reject(...) fails the turn fast, letting the existing fetch/retry path recover instead of hanging. Also feed.reject(...) if the stream ends without a message_stop/completion having been emitted.
2. Wire the request CancellationToken through — processResponseFromChatEndpoint and processResponseFromMessagesEndpoint are currently called without the cancellationToken that defaultChatResponseProcessor receives; pass it so user/turn cancellation can abort a stuck body read, and add a per-turn wall-clock deadline to catch dribbling keep-alive streams that never terminate.
3. Upstream/proxy — investigate why streaming responses can return 200 headers and then never stream a body or close; the client timeout is the safety net, not the root remedy.

Links

• Code: extensions/copilot/src/platform/endpoint/node/messagesApi.ts (processResponseFromMessagesEndpoint); extensions/copilot/src/platform/endpoint/node/responsesApi.ts (processResponseFromChatEndpoint)
• Eval analysis cache: ~/.agent/eval-cache/opus46-cli-compare/timeout-rootcause.md
• Related (eval-side, supersede — modal claim disproven): microsoft/vscode-copilot-evaluation#4863

[meganrogge]

Corroborating log timestamps (verbatim from GitHub Copilot Chat.log)

Pulled the exact lines from three hung instances across three runs. The signature is identical: a last successful claude-opus turn, a new turn issued that never logs a completion, then total silence until the OTel flush at exactly session-start + 60 min (the harness cap). The only intervening line is a background HMAC 401 model-metadata refetch ~55 min in — an effect of the wedge, not the cause.

circuit-fibsqrt — run 27511500953, opus-4.6

20:50:41.567  [GitExtensionServiceImpl] Initializing Git extension service.   <- session start
20:50:54.596  [messagesAPI] message 0 returned. finish reason: [stop]         <- last good turn (3rd)
20:50:54.598  ccreq:c5f81a8b | success | claude-opus-4.6 | 2094ms | [panel/editAgent]
   (4th /v1/messages turn issued — NO [messagesAPI] returned, NO ccreq ever logged)
21:45:41.554  [error] [CopilotCLISession] Failed to fetch models: 401 unable to validate HMAC   <- red herring, ~55 min in
21:50:42.694  [OTel] Flush complete                                            <- cap, exactly start+60:01

~55 min of dead silence between the stalled turn (20:50:54) and the next line (21:45:41).

regex-chess — run 27511511587, opus-4.7

21:07:04.274  Initializing Git extension service.                              <- session start
21:07:13.231  [messagesAPI] message 0 returned. finish reason: [stop]          <- last good turn
21:07:13.234  ccreq:bccd2bd6 | success | claude-opus-4.7 | 3275ms | [panel/editAgent]
   (next turn issued ~9s into the run — never returns)
22:02:04.306  [error] [CopilotCLISession] Failed to fetch models: 401 HMAC     <- red herring
22:07:05.200  [OTel] Flush complete                                            <- cap, exactly start+60:01

Hung on essentially the first agent turn (line 45 of the log) — ~54 min of silence.

winning-avg-corewars — run 27478622081, opus-4.6

21:42:52.850  Initializing Git extension service.                              <- session start
22:40:12.481  ccreq:7a6f72ba | success | claude-opus-4.6 | 133191ms | [panel/editAgent]
22:40:18.020  [messagesAPI] message 0 returned. finish reason: [stop]          <- last good turn (60th)
22:40:18.022  ccreq:8e64f54d | success | claude-opus-4.6 | 5439ms
   (next turn issued — never returns)
22:42:54.149  [OTel] Flush complete                                            <- cap, exactly start+60:01

This one ran 60 turns first, but note the per-turn latency instability on the same /v1/messages path: individual completed turns took 133,191ms (2.2 min) and elsewhere in the log 340,083 / 463,261 / 465,515 ms (up to 7.8 min). The path is slow and unstable even when it succeeds; sometimes it never terminates at all.

Takeaway

The "session-init → OTel-flush = exactly 60:00" delta on every hung instance confirms the entire wall-clock budget is burned after a single stalled streaming turn, with zero further log activity. There is no terminating message_stop, no error, no retry — the unguarded for await (const chunk of response.body) loop simply never returns.

[meganrogge]

Decisive proof the hang is a stalled streaming response (not a missing request, slow model, or hung tool)

The client chat log alone can't distinguish "turn never sent" from "turn sent but never returned." The CAPI proxy log — which sits between the client and the upstream — settles it. For circuit-fibsqrt (run 27511500953), the proxy issued 4 /v1/messages turns but the client only completed 3:

Proxy request	Proxy result	Client [messagesAPI] returned + ccreq
[7] 20:50:49.565	-> 200 + SSE headers	✅ 20:50:50.186 success (66a71c3d)
[8] 20:50:52.434	-> 200 + SSE headers	✅ 20:50:52.439 success (79e41389)
[9] 20:50:54.592	-> 200 + SSE headers	✅ 20:50:54.596 success (c5f81a8b)
[10] 20:50:57.527	-> 200 + SSE headers	❌ never — no return, no ccreq, no error, no retry

Raw proxy lines for the trailing turn:

[2026-06-14T20:50:57.527Z] [CAPI PROXY] [10] POST /v1/messages -> 200
[2026-06-14T20:50:57.527Z] [CAPI PROXY] [10] Response headers: cache-control, content-type, ... transfer-encoding, ... connection
[2026-06-14T20:50:57.527Z] [CAPI PROXY] [10] x-request-id: 2ef9a657-...

[10] is the last line in the proxy log — no [11], no body-complete marker, no error.

What this rules out

1. "The agent stopped issuing turns" — no. Proxy [10] POST /v1/messages proves turn 4 was dispatched at 20:50:57.527.
2. "Connection failure / non-200 the client rejected" — no. The proxy logged -> 200 plus full streaming Response headers (content-type SSE, transfer-encoding chunked). The response began.
3. "The model is just slow but will return" — no. Healthy turns [7][8][9] completed within ~0–4 ms of the proxy 200 (the ccreq "~2 s" is time-to-headers; the small SSE body then drained instantly). Turn [10] never returned and was killed by the harness cap ~59 min 45 s later.
4. "A tool call hung" — no. The final executed tool call completed and returned a result (see table below); the hang is the model turn issued after the tool result was sent back.

What it confirms

The client received 200 + SSE headers, then blocked inside for await (const chunk of response.body) because the body never streamed a terminating message_stop. With no idle timeout on that loop, no feed.emitOne / feed.reject ever fires, so the ToolCallingLoop await hangs until the 60-min wall-clock cap → X_AGENT_STILL_RESPONDING, score 0. The proxy going silent after [10]'s headers indicates the stall is in the body stream (upstream stopped mid-flight); the client's missing idle guard is what turns a recoverable stall into a full-budget hang.

Final executed tool call per hung instance (none of them hung — all returned)

Instance	Total tool calls	Final tool call (completed OK)	Time
circuit-fibsqrt	4 (all read_file)	read_file /app/sim.c L250–350	20:50:54.639Z
regex-chess	1	read_file /app/check.py L1–200	21:07:13.355Z
winning-avg-corewars	34 (18 run_in_terminal, 12 replace_string_in_file, 4 read_file)	run_in_terminal: pmars -b -r 100 -f my_warrior.red warriors/stone.red … paper.red …	22:40:19.279Z

In every case the final tool call's response field is populated (it completed), and the very next model streaming turn is the one that wedged — confirming the tool layer is not implicated.

[meganrogge]

Ruling out "the final tool call was just still running"

A reasonable alternative: maybe the agent wasn't hung on a model turn but on a tool that never finished (e.g. the pmars terminal run in winning-avg-corewars). The proxy log disproves this for all three hung instances. A new /v1/messages request is only dispatched after the previous tool's result has been collected and fed back to the model — so if a trailing proxy request exists after the final tool call, the tool must have returned.

In every case: proxy requests = client completions + 1, and that extra (final) proxy request is timestamped after the last tool call and is the one that never completes:

Instance	Final tool call	Trailing /v1/messages dispatched	proxy vs client completions
circuit-fibsqrt	20:50:54.639 (read_file)	[10] at 20:50:57.527 (+2.9s)	4 vs 3 (diff 1)
regex-chess	21:07:13.355 (read_file)	[8] at 21:07:17.062 (+3.7s)	2 vs 1 (diff 1)
winning-avg-corewars	22:40:19.279 (pmars run)	[96] at 22:40:23.360 (+4.1s)	61 vs 60 (diff 1)

If the tool were still in progress, the agent would block before issuing that trailing request, so proxy count would equal client count (diff 0) — not diff 1. For circuit-fibsqrt and regex-chess the point is even stronger: the final tool is read_file, a synchronous local file read that returns in milliseconds and whose response field is populated. It cannot be "in progress" for 60 minutes.

So the hang is unambiguously the model streaming turn issued after the tool returned, consuming the proxy's 200 + SSE headers and then never receiving a terminating message_stop — with no idle timeout to break the read loop.

[meganrogge]