SUZENT / Docs
THE SUZENT HANDBOOK

Stream recovery protocol

Desktop chat sends, retry/edit, steering, approval resume, heartbeat and Canvas turns can recover their in-progress output after a page reload or connection loss. Native and ACP pr

Desktop chat sends, retry/edit, steering, approval resume, heartbeat and Canvas turns can recover their in-progress output after a page reload or connection loss. Native and ACP producers share the same transport. The backend owns the snapshot; browser sessionStorage is only an optional visual seed.

Requests and frames

Start a queued turn using the existing /chat/send or /chat/steer-send endpoint, then POST /chat/live:

{"chat_id":"chat-id","protocol":1,"wait_ms":0}

For a direct /chat request, add "protocol":1. The response uses the same frames and its producer continues when the HTTP observer disconnects. Recovery always uses /chat/live, never repeats the original send.

The initial SSE frame contains an authoritative snapshot:

{"type":"STREAM_SNAPSHOT","run_id":"transport-run-id","seq":42,"events":[]}

events contains the current run's AG-UI events, compacting adjacent text, reasoning and tool-argument deltas. This deliberately reuses the frontend's existing reducer for Native and ACP custom events instead of introducing a second renderer-specific projection. Replace transient parts and reduce these events from an empty state. Previously completed turns remain in chat history. The transport run ID is distinct from provider/agent run IDs.

Subsequent frames contain one event and a monotonically increasing sequence:

{"type":"STREAM_EVENT","run_id":"transport-run-id","seq":43,"event":{"type":"TEXT_MESSAGE_CONTENT","messageId":"m","delta":"hello"}}

Reconnect with run_id and after_seq, the last applied sequence. Duplicate sequences are ignored. An unknown run, future cursor or expired replay window receives a replacement snapshot. Snapshot capture and subscription handoff occur on the same event loop without yielding; serialization may run on a worker. Events arriving during serialization are delivered after the snapshot cursor. Slow observers reset from a snapshot instead of silently losing an event suffix. Observers never compete for a consume-once queue.

Only the explicit terminal frame completes the transport:

{"type":"STREAM_END","run_id":"transport-run-id","seq":43,"persisted":true,"superseded":false}

Native streams wait for their own post-processing task's display/state commit; ACP already writes its response before its producer exits. A failed save sets persisted:false; it must not be presented as a successful save. Stale native revision finalizations are recorded as skipped, not successful writes. A replaced run sets superseded:true; the observer reconnects to the replacement run. RUN_FINISHED/AGENT_FINISHED inside the event stream do not mean the final display commit has completed. Neither does an unexpected HTTP EOF.

The desktop retains its transient response through the committed-history load, without appending a duplicate optimistic assistant message. Confirmed paths no longer use the fixed 800 ms delay. ACP approval resolutions and answered inline forms are included in the event history so replay does not reopen their actions.

An idle probe returns 204. A fresh probe of a completed, successfully saved run also returns 204 and the UI reloads history. Reconnects with a cursor can drain retained terminal events. A dropped event-bus subscriber receives an EOF so EventSource reconnects and refreshes its active-stream snapshot.

Model response waiting

The first-event prefill policy remains separate from subsequent event waits. Subsequent model waits resolve stream_idle_timeout_overrides by full model ID, then provider ID (including the canonical provider for aliases), then the provider's stream_idle_timeout_seconds in config/providers.json, then the global stream_idle_timeout_seconds. Tool-result waits retain their own deadlines. The Google provider ships a 300-second default; other providers fall back to 120 seconds. Local server activity probes can still extend a model wait.

After stream_wait_notice_seconds (15 by default) without a model event, the stream emits model_response_wait with active: true, stage waiting, and a unique wait_id. At stream_wait_slow_seconds (60 by default), stage slow offers Continue waiting when an extension is available. POST /chat/extend-wait requires chat_id, the active run_id or its client token, and the same wait_id. Stale runs and replaced waits return 409. An accepted extension wakes the pending wait and raises its deadline to stream_wait_extension_seconds (1,200 by default), measured from the start of this silence, not the button click. It is applied once per silent wait. Resumption, timeout, or cancellation clears the notice and active wait handle. The UI retains the existing Stop and Retry controls.

ObservedGoogleModel records SDK response chunk arrival gaps, response IDs, part counts, and finish reasons at DEBUG level before pydantic-ai converts the chunks to events. It never records text, tool arguments, or thought signatures. Chunks without model events therefore remain visible in diagnostics. This is SDK-chunk observation, not a packet-level network trace.

Tools declare display-only fields in model_excluded_metadata. ToolResultContextCapability removes those fields from copies of model-bound tool returns. Original tool events and persisted history retain their snapshots for display and rollback, including when a saved conversation is resumed.

Failed model requests

core/model_recovery.py sorts a run failure into a kind using pydantic-ai's error categories (with transport errors below the SDK matched by class name). Timeouts, dropped connections, rate limits and 5xx/overloaded responses are retried in the same stream after 2, 5 and 15 seconds, or after the provider's retry_after when it is longer (up to 60 seconds; a longer wait ends the run). A first-event timeout is retried only once, since each wait already lasts minutes. The retry count resets whenever a tool batch finishes. Authentication, quota, unknown model, context window and other rejected requests are not retried.

A retry resumes from deps.model_request_history, the request that failed, so finished tool results are kept and only the broken reply is requested again. Before it, the stream emits a model_recovery custom event (attempt, maxAttempts, reason, detail, delaySeconds). The AG-UI adapter first closes the part the failed response left open; the frontend reducer and the draft accumulator then drop that response's trailing text, reasoning and unfinished tool calls and append a model-notice part, which stays in the persisted reply.

When the run finally fails, RUN_ERROR.code names the kind (model_stalled, model_rate_limited, model_overloaded, model_auth, model_quota, model_not_found, model_context_exceeded) and the client shows a localized explanation with the provider's message as detail.

Auxiliary work after a reply

Automatic titles run as registered background tasks. Reply completion never waits for a pending title. Title generation has a shared ten-second budget across model attempts, then uses the local title heuristic. A completed title is persisted and announced with a chat_title_updated event on /events/stream.

Memory extraction is scheduled independently of the display/state commit. Work for the same chat is serialized, with a sixty-second execution timeout per turn. The bus emits memory_processing with chat_id and active; its initial snapshot includes memory_chats so reconnecting clients recover pending work. Status stays active until that chat's extraction queue drains, including failure or cancellation. The desktop shows this status near the input without keeping the reply's stop button active. STREAM_END{persisted:true} still waits for the reply's required persistence, but does not wait for auxiliary model requests.

Retention and limits

  • The delta replay window retains 4,096 events per run. The compact full snapshot is independent of that window and grows with the current run's output.
  • Adjacent deltas accumulate as fragments; full strings are joined only at an event boundary or snapshot read, avoiding quadratic per-token copying.
  • Completed runs are retained for up to five minutes and capped at 64 chats. Cleanup is lazy on registry reads/registration. Active producers and pending saves are not evicted. Existing connected observers keep their queue reference.
  • Recovery state is in the backend process. This supports browser reloads, chat switches, multiple observers and transient network loss. It does not introduce a durable event journal or resume an agent after a backend restart. After restart, persisted chat state remains the fallback.
  • Recovery retries use capped exponential backoff with five attempts after lost progress. Failed recovery preserves the received response and shows a localized error. This is not send idempotency: no automatic send retry is performed.
  • Existing clients omitting protocol retain their legacy wire format. The native mobile preview also uses protocol 1; its older independent replay implementation has been removed. Legacy multipart clients retain their old path.

Validation

Tests cover snapshot-to-live handoff, independently consuming observers, replay window overflow, invalid cursors, fragmented UTF-8 SSE, lost acknowledgements, unexpected EOF, duplicate deltas, replacement runs, save failure/cancellation, ACP approval resolution and resolved inline forms. Route tests disconnect a real response observer while a deterministic Native/ACP-shaped producer keeps running, then recover its prefix and suffix through /chat/live. These tests use isolated data and do not make paid model calls or modify real conversations.

A synthetic transport check on the development Mac appended 10,000 one-KiB text deltas (9.77 MiB) in 38.74 ms total, captured the compact snapshot in 0.96 ms, and retained one snapshot event plus 4,096 tail events. This measures the replay accumulator only; it excludes model latency, networking and browser rendering.

Edit this page on GitHub ↗

On this page