SUZENT / Docs
THE SUZENT HANDBOOK

Model capabilities and roles

This page is for contributors. For choosing providers and assigning model roles as a user, see Models & Providers. Suzent keeps a registry of per-model metadata — context window, c

This page is for contributors. For choosing providers and assigning model roles as a user, see Models & Providers.

Suzent keeps a registry of per-model metadata — context window, capability flags (vision, function calling, reasoning, prompt caching), and pricing. This data drives context compression, role routing, cost estimation, and what the UI shows for each model.

The registry is loaded from three layers, each overlaying the previous one:

LayerPathTracked in git?Written by
1. Shipped defaultsconfig/capabilities/{provider}.jsonYesMaintainers (curated)
2. Local overlay<data dir>/capabilities/{provider}.jsonNoRuntime discovery
3. Global overridesconfig/model_capabilities.jsonYesMaintainers (applied last)

<data dir> defaults to ~/.suzent (override with SUZENT_DATA_DIR).

For a model ID present in more than one layer, the shipped curated entry wins over the local overlay — the overlay only supplies models that aren't shipped. The global override file is applied last and takes precedence over everything.

Why the overlay exists

The app discovers models at runtime — when you click FETCH for a provider, and via a periodic LiteLLM sync that refreshes context windows and pricing. Those writes go to the local overlay, never to the tracked config/capabilities/ files.

This keeps the repo clean: stable suzent update checks out an exact release, while suzent update --dev fast-forwards main. If runtime discovery had been writing into tracked files, either update could conflict. With the overlay, discovered models persist across updates in your data directory while the shipped files stay pristine. For safety, the updater discards stale local edits under config/capabilities/ before changing revisions.

The overlay is auto-generated and safe to delete; it will be repopulated on the next discovery.

Updating the repository data

Developer mode follows the same rule as normal runtime: provider FETCH, LiteLLM sync, and stale-model pruning write to the local overlay. Running suzent start --dev therefore does not modify tracked capability files.

If you maintain Suzent and want to refresh the tracked files, use the dedicated maintenance command:

uv run python scripts/sync_model_capabilities.py --to-repo

This explicitly enables SUZENT_CAPABILITIES_TO_REPO=1 for that process. Review the generated diff before committing it. The scheduled Update Model Capabilities workflow uses this command to open or update a dedicated pull request.

Adding a model by hand

To curate a model permanently, add it to its provider file in config/capabilities/. The minimal entry is just a mode; fill in the rest to improve context-window and cost accuracy:

{
  "models": {
    "anthropic/claude-opus-4-8": {
      "mode": "chat",
      "max_input_tokens": 200000,
      "max_output_tokens": 32000,
      "supports_vision": true,
      "supports_function_calling": true,
      "supports_reasoning": true,
      "supports_prompt_caching": true,
      "supports_response_schema": true
    }
  }
}

mode is one of chat, embedding, image_generation, or tts. Keys starting with _ (e.g. _doc) are treated as comments and ignored.

Image editing

The registry preserves LiteLLM's supported_endpoints and image_edit mode. A model is confirmed to support editing when its endpoints include /v1/images/edits or its mode is image_edit. Missing metadata means unknown, not unsupported; some providers' endpoint lists are incomplete. Runtime sync and scripts/sync_model_capabilities.py use the same parser. Bare OpenAI model IDs are normalized to the openai/ prefix used by Suzent.

Configure Image Editing separately in Settings → Model Roles. Confirmed models appear as suggestions; enabled models with unknown editing capability remain available as explicit overrides. There is no chat-model fallback.

The edit_image tool accepts a prompt, local image_paths, optional PNG mask_path, size, quality, and count (1–4). Each input is limited to 20 MB; providers can impose stricter limits. Mask dimensions must match the source. Multi-image, mask, dimensions, and quality support depend on the selected model. Results are saved as new files in the workspace's images directory; pass the returned saved_paths to edit them again. Existing chats with a saved tool selection may need ImageEditTool enabled.

generate_image passes size and quality to the image API and requests the count as a native batch. DALL-E 3 therefore requires count=1. Its free-form style remains a prompt hint. Both tools save all returned images with extensions identified from the image bytes and report empty provider responses as errors.

When a provider does not support quality, image tools omit that optional hint and report dropped_params: ["quality"] along with a note in the result. Unsupported size, mask, and batch-count parameters still fail explicitly.

Model role inheritance

Settings → Model Roles assigns models by task. An empty task role inherits the nearest configured parent, including its ordered model list:

primary
├── cheap
│   ├── title
│   ├── suggestion
│   ├── memory_extraction
│   └── decision
│       ├── goal_judge
│       └── permission_review
└── dream
  • title: conversation titles.
  • suggestion: next-message prediction for the composer's ghost text (POST /chat/input-suggestion; the client sends its recent messages because the turn's final save can land after the stream ends). With chat_id and reply_key the prediction is stored in the input_suggestions table, and GET /chat/input-suggestion?chat_id= returns it so any device can restore it while the chat's last reply still matches reply_key.
  • memory_extraction: extract durable facts from conversations.
  • decision: shared default for goal completion and automatic permission review.
  • goal_judge / permission_review: optional overrides under Advanced decision roles.
  • dream: memory consolidation and knowledge-vault lint passes.

Explicit assignments replace inheritance; clearing a task role restores it, including after restart when YAML defines a default. Saving roles updates the running memory extractor immediately; an extraction already in progress keeps its original client. Inheritance selects configuration, not a second retry path after a model fails. Individual callers retain their existing retry behavior. Context compaction still uses the conversation model.

Existing cheap configurations continue to serve title, extraction, and decision tasks. A saved role assignment takes precedence over YAML role defaults. The legacy memory_consolidation_model setting supplies the initial Dream assignment when no explicit dream role exists; Dream otherwise inherits Primary. An explicitly cleared Dream role remains empty on restart, even when the legacy setting is still present.

Vision retains its capability-filtered inheritance from Primary. Embedding, image generation, and TTS require explicit specialist models and have no parent role.

Provider registry

Built-in providers are declared in config/providers.json. User-defined providers are merged from config/providers.user.json and appear alongside the built-in cards after the registry reloads. The ChatGPT Subscription provider (chatgpt/... model IDs) authenticates through LiteLLM's ChatGPT device-code flow; its tokens live in Suzent's local config directory and are never returned by the HTTP status endpoint.

Decision models

A decision model answers typed questions instead of writing text. It is built by create_decision_model() in core/model_factory.py from the provider's decisions_protocol in the provider registry:

decisions_protocolEndpointpydantic-ai classShipped providers
openai/v1/decisionsOpenAIDecisionsModelopenai, litellm_proxy
geminiGemini Decisions APIGoogleDecisionsModelgemini
system_one/v1/systemoneSystemOneModelnone; add one in providers.user.json

The role picker suggests catalog models whose supported_endpoints include /v1/decisions, and drops models whose only endpoint is that one from every chat role. Any other model ID can still be typed in.

A System One provider, for TypeSafe's Jev or a self-hosted server, goes in config/providers.user.json:

{
  "providers": [
    {
      "id": "typesafe",
      "label": "TypeSafe",
      "api_type": "openai",
      "base_url": "https://api.typesafe.ai",
      "decisions_protocol": "system_one",
      "env_keys": ["TYPESAFE_API_KEY"],
      "fields": [{"key": "TYPESAFE_API_KEY", "label": "API Key", "type": "secret"}]
    }
  ]
}

and is then chosen as typesafe/jev-latest. Set "api_key_optional": true for a local server without a key.

Prompt caching

Long agent runs resend the same prefix (tools, instructions, history) on every request, so cache hits decide most of the input cost. Providers differ in what the client has to do:

ProviderHow caching is enabledWhere Suzent sets it
Gemini, DeepSeek, local serversAutomatic prefix caching on the serverNothing to send
AnthropicOnly when the request marks cache_control breakpointsanthropic_cache_settings() in core/model_factory.py: tool and instruction breakpoints plus top-level automatic caching. Endpoints with a custom base_url get a last-message breakpoint instead, since compatible gateways may not implement automatic caching
OpenAI APIAutomatic, routed by prompt_cache_keyPromptCacheRoutingCapability in core/prompt_cache.py sets the key to the chat id on every request
ChatGPT SubscriptionAutomatic, routed by the session_id headerSame capability, as a session_id header carrying the chat id

The routing key has to be the chat being served, not a value fixed on the model: agents are cached and shared across chats. Without it, consecutive requests of one chat land on machines that never saw its prefix, and hits swing between almost all and almost none on a context that only grew.

cost_ledger records one row per model request with cache_read_tokens and cache_write_tokens, which is the place to check hit rates per model or chat.

The prefix is ordered tool definitions, then the system prompt, then the history. Anything that changes early in that order makes the provider re-read everything after it at full price, so the runtime keeps the prefix byte-identical from one request to the next:

  • Core memory is pinned per chat. The system prompt carries the core-memory version the chat started with (memory/core_memory_pin.py). Later edits to persona, user, MEMORY.md or the project's context.md reach the model once, as a diff in a system reminder on the next turn, which lands after the cached history. A diff counts as seen only if it survives the reminder budget; otherwise it is offered again next turn. The pin follows the live files only when that costs nothing extra: a chat with no reply yet, after compaction, after a restart, or when the diff is larger than MAX_DIFF_CHARS. Pins are keyed by chat and user and never held on the shared agent instance.
  • Tool-result trimming moves in steps. ToolResultTrimmer trims results older than a boundary that advances TRIM_STEP messages at a time, and trimming an already trimmed result is a no-op. The trimmed history is what the run persists, so most turns resend the earlier prefix unchanged. A hard clear keeps the exact cutoff: it runs at the compaction threshold, where freeing space matters more than the cache.
  • Discovered tools depend on the provider profile. With the local search_tools fallback, pydantic-ai adds a discovered tool's definition to the tools array on providers whose profile has no tool_addition_mode (Gemini, Chat Completions, OpenAI-compatible endpoints), which invalidates the whole prefix once per discovery. First-party OpenAI Responses, including ChatGPT, uses with_definitions and appends the definition after the history instead. Closing the gap for the other providers needs an upstream wire format, or sending every deferred schema up front.

Compaction summaries

A compaction summary is written by repeating the chat's latest request with one instruction added at the end of its last message, so the provider serves the whole history from cache instead of a separate model re-reading a flattened transcript at full price (summarize_on_cached_prefix() in core/context_compressor.py).

  • The exact request. RequestRecorderCapability (core/prompt_cache.py, registered last) wraps the model that actually sends, and records each chat's latest request: model, settings, parameters, and the messages after pydantic-ai prepared them (framing, tool-search exchanges, the system voice announcement). Repeating the stored history instead would differ from the wire and miss the cache. It also records the agent history the request was prepared from, and the summary is only written that way when that history covers everything being folded (compared by history_fingerprint(), which ignores tool output so trimming between turns does not break the match).
  • Nothing else changes. Tools stay declared and tool_choice stays as it was, because changing either invalidates Anthropic's cached messages; the instruction tells the model not to call tools, and a reply that does is retried once with a correction.
  • Fallback. No recorded request (first request after a restart), a history over 95% of the window, a provider error or an unusable reply all fall back to the extraction model on a flattened transcript. compaction_reuse_prompt_cache: false turns the cached path off.
  • Thresholds. compaction_trigger_tokens() and precompaction_trigger_tokens() take the smaller of a share of the window and a fixed token count: by default 80% or 500k for compaction, 50% or 350k for preparing a summary. The fixed count matters on 1M windows: published multi-needle retrieval results (MRCR v2) for the strongest models stay around 80% up to about 500k and fall to the 70s or lower toward 1M, while mid-tier models fall off near 200k.
  • Prepared in the background. schedule_precompaction() starts the next summary while the cache is warm, from two places: ChatProcessor when a turn finishes, and the history processor before each request of a run that is past the precompaction threshold but under the trigger, when prefix_cache_warm() says the chat's last request is likely still cached. The summary is held in _PRECOMPACTIONS, not applied. When the trigger is reached, the history processor applies it if its fingerprint still matches, waits for it if it is still being written, and writes a fresh one if it no longer matches or would leave the context over the trigger. A summary that still covers the history, or one that failed, is not rewritten until the uncovered tail passes 20% of the compaction trigger (PRECOMPACTION_REFRESH_TAIL_SHARE).
  • Kept across restarts. A finished summary is written to the prepared_summaries table (chat id, end_index, fingerprint, text) and deleted when it is applied or superseded. The first time the history processor sees a chat past the precompaction threshold in a process, restore_precompaction() loads it back. With no request recorded since the restart, the cold-cache check measures idle time from the last reply's timestamp.
  • Applied early on a cold cache. Each RequestSnapshot carries the Suzent model id and the time it was sent. Before the trigger is reached, cold_cache_compaction_due() applies a finished prepared summary when the history is at or over the precompaction threshold and the time since the chat's last request exceeds the cache TTL estimate: the next request would pay full input price for the whole history anyway. Providers with no TTL estimate (local models) never qualify.
  • Choosing the writer. ContextCompressor._prefer_cached_prefix() keeps the cached path while the cache is likely warm. Once it has likely expired, it prices both options with estimate_cost_usd() (genai-prices, the same source as the cost ledger): the conversation's model on the whole prepared request billed as a cache write, against the extraction model on the folded messages, each with a typical summary's output. The cheaper one writes it; if either model is unpriced, the cached path is kept. The choice and both estimates are logged at INFO.

Events and notices

Every pass emits an auto_compaction bus event (start, then complete, skipped or error). A completed one carries reason: prepared when a background summary was applied at the trigger, cold_cache when it was applied early (with idle_seconds), or fresh when it was written on the spot, plus the summary text. The history processor and /compact persist the terminal notice with those fields in its metadata, which the chat renders as a divider (CompactionDivider) and the context panel reads for its "last compacted" line. schedule_precompaction() emits a precompaction event (start, then ready or failed, with covered_messages) so the panel can show the summary being prepared.

Cache TTL estimate

No provider reports how long it keeps a cached prefix, so core/cache_ttl.py estimates it per model:

  • Anthropic uses the TTL the request asked for: 5 minutes, or an hour with anthropic_prompt_cache_ttl: "1h", which anthropic_cache_settings() sets on every breakpoint (a longer-lived breakpoint may not follow a shorter one).
  • Other providers start from PROVIDER_DEFAULT_TTL_SECONDS: OpenAI, ChatGPT and Gemini 300s, DeepSeek 3600s. Anything else is unknown.
  • Observed hits raise the estimate. The recording wrapper reports every response to observe_request(); a request that read at least 1,024 cached tokens after the model had been idle (in any chat) for 60s or more proves the cache lives at least that long. The longest such gap per model is kept in <data dir>/prompt_cache_ttl.json. Misses never lower it, since a changed prefix misses too. The cost ledger is not used for this: it stores one row per turn, summing every request in it, so it cannot tell whether the first request after a pause hit the cache.

Forked sub-agents

A sub-agent spawned with inherit_context=True starts from its parent's history, so it can read the parent's cached prefix, but only if the tools and system prompt in front of that history are byte-identical. A sub-agent built the ordinary way gets its own system prompt and a narrowed tool list, which misses the cache on its first request. A fork is built to match instead (fork_turn_config() in core/subagent_runner.py):

  • Same config. ChatProcessor records the config each chat's current turn built its agent from (record_turn_config()). The fork copies its parent's, so it resolves to the same cached agent and the same tool definitions, and overrides only keys that change neither: permission mode, interaction profile, memory, working directory and grants.
  • Same instructions. ForkPrefixCapability snapshots the instruction parts of every request. At spawn the parent's latest snapshot is pinned to the fork, and every request the fork makes, follow-ups included, sends those parts instead of its own. Recomputing them would differ in the chat id, permission mode text and memory pin.
  • Role as a reminder. The sub-agent role, its tool set and the rules that outrank the inherited prompt (no spawning, no questions to the user) go in a hidden system reminder with the task (SUBAGENT_FORK_REMINDER), after the cached history.
  • Tools refused at call time. The fork keeps every tool definition, and ToolDenyCapability.before_tool_validate fails a call outside call_allowed_tools. Builtins and MCP tools stay callable, as they are for an ordinary sub-agent. Without an explicit tool list the allowlist is the parent's tools, minus the agent tools, so a fork cannot spawn sub-agents.
  • Same routing key. prompt_cache_chat_id makes PromptCacheRoutingCapability key the fork by its parent's chat, so OpenAI and ChatGPT route both to the machine holding the prefix.

The fork falls back to an ordinary sub-agent when the parent has no turn recorded in this process (after a restart) or the fork runs another model. A pinned fork logs its tool-definition drift from the parent, if any, and its cache read share for every request.

Set SUZENT_CACHE_DIAGNOSTICS=1 to also send anthropic_cache_diagnostics on requests to api.anthropic.com, so a miss is explained by the first point where the prefix diverged. Anthropic compares a request only with an earlier one that also opted in, so the parent's requests carry the flag too. It is off by default because HIPAA-enabled organizations reject it, and other Anthropic endpoints do not support it. OpenAI's Responses API reports its own diagnostics without a flag, and the fork log includes either.

For every other chat, ForkPrefixCapability logs two DEBUG lines per model request. The first says what changed in the prefix since that chat's previous request: the tool definitions, or the first instruction section whose hash moved, named by its first line (section 12 '# Memory System' changed (c85e47c9->ccad99b0)). The second gives the cache read share. A miss with unchanged before it points at the provider (routing, expiry), not at the prompt. Hashes live in this process only, so the first request after a restart reads first request in this process.

Edit this page on GitHub ↗

On this page