Stream recovery protocol
Desktop chat sends, retry/edit, steering, approval resume, heartbeat and Canvas turns can recover their in-progress output after a page reload or connection loss. Native and ACP pr
Desktop chat sends, retry/edit, steering, approval resume, heartbeat and Canvas turns can recover their in-progress output after a page reload or connection loss. Native and ACP producers share the same transport. The backend owns the snapshot; browser sessionStorage is only an optional visual seed.
Requests and frames
Start a queued turn using the existing /chat/send or /chat/steer-send endpoint,
then POST /chat/live:
{"chat_id":"chat-id","protocol":1,"wait_ms":0}For a direct /chat request, add "protocol":1. The response uses the same
frames and its producer continues when the HTTP observer disconnects. Recovery
always uses /chat/live, never repeats the original send.
The initial SSE frame contains an authoritative snapshot:
{"type":"STREAM_SNAPSHOT","run_id":"transport-run-id","seq":42,"events":[]}events contains the current run's AG-UI events, compacting adjacent text,
reasoning and tool-argument deltas. This deliberately reuses the frontend's
existing reducer for Native and ACP custom events instead of introducing a
second renderer-specific projection. Replace transient parts and reduce these
events from an empty state. Previously completed turns remain in chat history.
The transport run ID is distinct from provider/agent run IDs.
Subsequent frames contain one event and a monotonically increasing sequence:
{"type":"STREAM_EVENT","run_id":"transport-run-id","seq":43,"event":{"type":"TEXT_MESSAGE_CONTENT","messageId":"m","delta":"hello"}}Reconnect with run_id and after_seq, the last applied sequence. Duplicate
sequences are ignored. An unknown run, future cursor or expired replay window
receives a replacement snapshot. Snapshot capture and subscription handoff occur
on the same event loop without yielding; serialization may run on a worker.
Events arriving during serialization are delivered after the snapshot cursor.
Slow observers reset from a snapshot instead of silently losing an event suffix.
Observers never compete for a consume-once queue.
Only the explicit terminal frame completes the transport:
{"type":"STREAM_END","run_id":"transport-run-id","seq":43,"persisted":true,"superseded":false}Native streams wait for their own post-processing task's display/state commit;
ACP already writes its response before its producer exits. A failed save sets
persisted:false; it must not be presented as a successful save. Stale native
revision finalizations are recorded as skipped, not successful writes. A replaced
run sets superseded:true; the observer reconnects to the replacement run.
RUN_FINISHED/AGENT_FINISHED inside the event stream do not mean the final
display commit has completed. Neither does an unexpected HTTP EOF.
The desktop retains its transient response through the committed-history load, without appending a duplicate optimistic assistant message. Confirmed paths no longer use the fixed 800 ms delay. ACP approval resolutions and answered inline forms are included in the event history so replay does not reopen their actions.
An idle probe returns 204. A fresh probe of a completed, successfully saved run also returns 204 and the UI reloads history. Reconnects with a cursor can drain retained terminal events. A dropped event-bus subscriber receives an EOF so EventSource reconnects and refreshes its active-stream snapshot.
Model response waiting
The first-event prefill policy remains separate from subsequent event waits.
Subsequent model waits resolve stream_idle_timeout_overrides by full model ID,
then provider ID (including the canonical provider for aliases), then the provider's
stream_idle_timeout_seconds in config/providers.json, then the global
stream_idle_timeout_seconds. Tool-result waits retain their own deadlines.
The Google provider ships a 300-second default; other providers fall back to
120 seconds. Local server activity probes can still extend a model wait.
After stream_wait_notice_seconds (15 by default) without a model event, the
stream emits model_response_wait with active: true, stage waiting, and a
unique wait_id. At stream_wait_slow_seconds (60 by default), stage slow
offers Continue waiting when an extension is available. POST /chat/extend-wait
requires chat_id, the active run_id or its client token, and the same wait_id.
Stale runs and replaced waits return 409. An accepted extension wakes the pending
wait and raises its deadline to stream_wait_extension_seconds (1,200 by default),
measured from the start of this silence, not the button click. It is applied once
per silent wait. Resumption, timeout, or cancellation clears the notice and active
wait handle. The UI retains the existing Stop and Retry controls.
ObservedGoogleModel records SDK response chunk arrival gaps, response IDs,
part counts, and finish reasons at DEBUG level before pydantic-ai converts the
chunks to events. It never records text, tool arguments, or thought signatures.
Chunks without model events therefore remain visible in diagnostics. This is
SDK-chunk observation, not a packet-level network trace.
Tools declare display-only fields in model_excluded_metadata.
ToolResultContextCapability removes those fields from copies of model-bound
tool returns. Original tool events and persisted history retain their snapshots
for display and rollback, including when a saved conversation is resumed.
Failed model requests
core/model_recovery.py sorts a run failure into a kind using pydantic-ai's
error categories (with transport errors below the SDK matched by class name).
Timeouts, dropped connections, rate limits and 5xx/overloaded responses are
retried in the same stream after 2, 5 and 15 seconds, or after the provider's
retry_after when it is longer (up to 60 seconds; a longer wait ends the run).
A first-event timeout is retried only once, since each wait already lasts
minutes. The retry count resets whenever a tool batch finishes. Authentication,
quota, unknown model, context window and other rejected requests are not retried.
A retry resumes from deps.model_request_history, the request that failed, so
finished tool results are kept and only the broken reply is requested again.
Before it, the stream emits a model_recovery custom event (attempt,
maxAttempts, reason, detail, delaySeconds). The AG-UI adapter first
closes the part the failed response left open; the frontend reducer and the
draft accumulator then drop that response's trailing text, reasoning and
unfinished tool calls and append a model-notice part, which stays in the
persisted reply.
When the run finally fails, RUN_ERROR.code names the kind (model_stalled,
model_rate_limited, model_overloaded, model_auth, model_quota,
model_not_found, model_context_exceeded) and the client shows a localized
explanation with the provider's message as detail.
Auxiliary work after a reply
Automatic titles run as registered background tasks. Reply completion never waits
for a pending title. Title generation has a shared ten-second budget across model
attempts, then uses the local title heuristic. A completed title is persisted and
announced with a chat_title_updated event on /events/stream.
Memory extraction is scheduled independently of the display/state commit. Work
for the same chat is serialized, with a sixty-second execution timeout per turn.
The bus emits memory_processing with chat_id and active; its initial snapshot
includes memory_chats so reconnecting clients recover pending work. Status stays
active until that chat's extraction queue drains, including failure or cancellation.
The desktop shows this status near the input without keeping the reply's stop
button active. STREAM_END{persisted:true} still waits for the reply's required
persistence, but does not wait for auxiliary model requests.
Retention and limits
- The delta replay window retains 4,096 events per run. The compact full snapshot is independent of that window and grows with the current run's output.
- Adjacent deltas accumulate as fragments; full strings are joined only at an event boundary or snapshot read, avoiding quadratic per-token copying.
- Completed runs are retained for up to five minutes and capped at 64 chats. Cleanup is lazy on registry reads/registration. Active producers and pending saves are not evicted. Existing connected observers keep their queue reference.
- Recovery state is in the backend process. This supports browser reloads, chat switches, multiple observers and transient network loss. It does not introduce a durable event journal or resume an agent after a backend restart. After restart, persisted chat state remains the fallback.
- Recovery retries use capped exponential backoff with five attempts after lost progress. Failed recovery preserves the received response and shows a localized error. This is not send idempotency: no automatic send retry is performed.
- Existing clients omitting
protocolretain their legacy wire format. The native mobile preview also uses protocol 1; its older independent replay implementation has been removed. Legacy multipart clients retain their old path.
Validation
Tests cover snapshot-to-live handoff, independently consuming observers, replay
window overflow, invalid cursors, fragmented UTF-8 SSE, lost acknowledgements,
unexpected EOF, duplicate deltas, replacement runs, save failure/cancellation,
ACP approval resolution and resolved inline forms. Route tests disconnect a
real response observer while a deterministic Native/ACP-shaped producer keeps
running, then recover its prefix and suffix through /chat/live. These tests
use isolated data and do not make paid model calls or modify real conversations.
A synthetic transport check on the development Mac appended 10,000 one-KiB text deltas (9.77 MiB) in 38.74 ms total, captured the compact snapshot in 0.96 ms, and retained one snapshot event plus 4,096 tail events. This measures the replay accumulator only; it excludes model latency, networking and browser rendering.