Model capabilities and roles
This page is for contributors. For choosing providers and assigning model roles as a user, see Models & Providers. Suzent keeps a registry of per-model metadata — context window, c
This page is for contributors. For choosing providers and assigning model roles as a user, see Models & Providers.
Suzent keeps a registry of per-model metadata — context window, capability flags (vision, function calling, reasoning, prompt caching), and pricing. This data drives context compression, role routing, cost estimation, and what the UI shows for each model.
The registry is loaded from three layers, each overlaying the previous one:
| Layer | Path | Tracked in git? | Written by |
|---|---|---|---|
| 1. Shipped defaults | config/capabilities/{provider}.json | Yes | Maintainers (curated) |
| 2. Local overlay | <data dir>/capabilities/{provider}.json | No | Runtime discovery |
| 3. Global overrides | config/model_capabilities.json | Yes | Maintainers (applied last) |
<data dir> defaults to ~/.suzent (override with SUZENT_DATA_DIR).
For a model ID present in more than one layer, the shipped curated entry wins over the local overlay — the overlay only supplies models that aren't shipped. The global override file is applied last and takes precedence over everything.
Why the overlay exists
The app discovers models at runtime — when you click FETCH for a provider,
and via a periodic LiteLLM sync that refreshes context windows
and pricing. Those writes go to the local overlay, never to the tracked
config/capabilities/ files.
This keeps the repo clean: stable suzent update checks out an exact release,
while suzent update --dev fast-forwards main. If runtime discovery had been
writing into tracked files, either update could conflict. With the overlay,
discovered models persist across updates in your data directory while the
shipped files stay pristine. For safety, the updater discards stale local edits
under config/capabilities/ before changing revisions.
The overlay is auto-generated and safe to delete; it will be repopulated on the next discovery.
Updating the repository data
Developer mode follows the same rule as normal runtime: provider FETCH,
LiteLLM sync, and stale-model pruning write to the local overlay. Running
suzent start --dev therefore does not modify tracked capability files.
If you maintain Suzent and want to refresh the tracked files, use the dedicated maintenance command:
uv run python scripts/sync_model_capabilities.py --to-repoThis explicitly enables SUZENT_CAPABILITIES_TO_REPO=1 for that process.
Review the generated diff before committing it. The scheduled
Update Model Capabilities workflow uses this command to open or update a
dedicated pull request.
Adding a model by hand
To curate a model permanently, add it to its provider file in
config/capabilities/. The minimal entry is just a mode; fill in the rest to
improve context-window and cost accuracy:
{
"models": {
"anthropic/claude-opus-4-8": {
"mode": "chat",
"max_input_tokens": 200000,
"max_output_tokens": 32000,
"supports_vision": true,
"supports_function_calling": true,
"supports_reasoning": true,
"supports_prompt_caching": true,
"supports_response_schema": true
}
}
}mode is one of chat, embedding, image_generation, or tts. Keys
starting with _ (e.g. _doc) are treated as comments and ignored.
Image editing
The registry preserves LiteLLM's supported_endpoints and image_edit mode.
A model is confirmed to support editing when its endpoints include
/v1/images/edits or its mode is image_edit. Missing metadata means unknown,
not unsupported; some providers' endpoint lists are incomplete. Runtime sync and
scripts/sync_model_capabilities.py use the same parser. Bare OpenAI model IDs
are normalized to the openai/ prefix used by Suzent.
Configure Image Editing separately in Settings → Model Roles. Confirmed models appear as suggestions; enabled models with unknown editing capability remain available as explicit overrides. There is no chat-model fallback.
The edit_image tool accepts a prompt, local image_paths, optional PNG
mask_path, size, quality, and count (1–4). Each input is limited to 20 MB;
providers can impose stricter limits. Mask dimensions must match the source.
Multi-image, mask, dimensions, and quality support depend on the selected model.
Results are saved as new files in the workspace's images directory; pass the
returned saved_paths to edit them again. Existing chats with a saved tool
selection may need ImageEditTool enabled.
generate_image passes size and quality to the image API and requests the count
as a native batch. DALL-E 3 therefore requires count=1. Its free-form style
remains a prompt hint. Both tools save all returned images with extensions
identified from the image bytes and report empty provider responses as errors.
When a provider does not support quality, image tools omit that optional hint
and report dropped_params: ["quality"] along with a note in the result.
Unsupported size, mask, and batch-count parameters still fail explicitly.
Model role inheritance
Settings → Model Roles assigns models by task. An empty task role inherits the nearest configured parent, including its ordered model list:
primary
├── cheap
│ ├── title
│ ├── suggestion
│ ├── memory_extraction
│ └── decision
│ ├── goal_judge
│ └── permission_review
└── dreamtitle: conversation titles.suggestion: next-message prediction for the composer's ghost text (POST /chat/input-suggestion; the client sends its recent messages because the turn's final save can land after the stream ends). Withchat_idandreply_keythe prediction is stored in theinput_suggestionstable, andGET /chat/input-suggestion?chat_id=returns it so any device can restore it while the chat's last reply still matchesreply_key.memory_extraction: extract durable facts from conversations.decision: shared default for goal completion and automatic permission review.goal_judge/permission_review: optional overrides under Advanced decision roles.dream: memory consolidation and knowledge-vault lint passes.
Explicit assignments replace inheritance; clearing a task role restores it, including after restart when YAML defines a default. Saving roles updates the running memory extractor immediately; an extraction already in progress keeps its original client. Inheritance selects configuration, not a second retry path after a model fails. Individual callers retain their existing retry behavior. Context compaction still uses the conversation model.
Existing cheap configurations continue to serve title, extraction, and decision
tasks. A saved role assignment takes precedence over YAML role defaults. The legacy
memory_consolidation_model setting supplies the initial Dream assignment when
no explicit dream role exists; Dream otherwise inherits Primary. An explicitly cleared Dream role remains empty on restart, even when the legacy
setting is still present.
Vision retains its capability-filtered inheritance from Primary. Embedding, image generation, and TTS require explicit specialist models and have no parent role.
Provider registry
Built-in providers are declared in config/providers.json. User-defined
providers are merged from config/providers.user.json and appear alongside the
built-in cards after the registry reloads. The ChatGPT Subscription provider
(chatgpt/... model IDs) authenticates through LiteLLM's ChatGPT device-code
flow; its tokens live in Suzent's local config directory and are never returned
by the HTTP status endpoint.
Decision models
A decision model answers typed questions instead of writing text. It is built
by create_decision_model() in core/model_factory.py from the provider's
decisions_protocol in the provider registry:
decisions_protocol | Endpoint | pydantic-ai class | Shipped providers |
|---|---|---|---|
openai | /v1/decisions | OpenAIDecisionsModel | openai, litellm_proxy |
gemini | Gemini Decisions API | GoogleDecisionsModel | gemini |
system_one | /v1/systemone | SystemOneModel | none; add one in providers.user.json |
The role picker suggests catalog models whose supported_endpoints include
/v1/decisions, and drops models whose only endpoint is that one from every chat
role. Any other model ID can still be typed in.
A System One provider, for TypeSafe's Jev or a self-hosted server, goes in
config/providers.user.json:
{
"providers": [
{
"id": "typesafe",
"label": "TypeSafe",
"api_type": "openai",
"base_url": "https://api.typesafe.ai",
"decisions_protocol": "system_one",
"env_keys": ["TYPESAFE_API_KEY"],
"fields": [{"key": "TYPESAFE_API_KEY", "label": "API Key", "type": "secret"}]
}
]
}and is then chosen as typesafe/jev-latest. Set "api_key_optional": true for a
local server without a key.
Prompt caching
Long agent runs resend the same prefix (tools, instructions, history) on every request, so cache hits decide most of the input cost. Providers differ in what the client has to do:
| Provider | How caching is enabled | Where Suzent sets it |
|---|---|---|
| Gemini, DeepSeek, local servers | Automatic prefix caching on the server | Nothing to send |
| Anthropic | Only when the request marks cache_control breakpoints | anthropic_cache_settings() in core/model_factory.py: tool and instruction breakpoints plus top-level automatic caching. Endpoints with a custom base_url get a last-message breakpoint instead, since compatible gateways may not implement automatic caching |
| OpenAI API | Automatic, routed by prompt_cache_key | PromptCacheRoutingCapability in core/prompt_cache.py sets the key to the chat id on every request |
| ChatGPT Subscription | Automatic, routed by the session_id header | Same capability, as a session_id header carrying the chat id |
The routing key has to be the chat being served, not a value fixed on the model: agents are cached and shared across chats. Without it, consecutive requests of one chat land on machines that never saw its prefix, and hits swing between almost all and almost none on a context that only grew.
cost_ledger records one row per model request with cache_read_tokens and
cache_write_tokens, which is the place to check hit rates per model or chat.
The prefix is ordered tool definitions, then the system prompt, then the history. Anything that changes early in that order makes the provider re-read everything after it at full price, so the runtime keeps the prefix byte-identical from one request to the next:
- Core memory is pinned per chat. The system prompt carries the core-memory
version the chat started with (
memory/core_memory_pin.py). Later edits to persona, user,MEMORY.mdor the project'scontext.mdreach the model once, as a diff in a system reminder on the next turn, which lands after the cached history. A diff counts as seen only if it survives the reminder budget; otherwise it is offered again next turn. The pin follows the live files only when that costs nothing extra: a chat with no reply yet, after compaction, after a restart, or when the diff is larger thanMAX_DIFF_CHARS. Pins are keyed by chat and user and never held on the shared agent instance. - Tool-result trimming moves in steps.
ToolResultTrimmertrims results older than a boundary that advancesTRIM_STEPmessages at a time, and trimming an already trimmed result is a no-op. The trimmed history is what the run persists, so most turns resend the earlier prefix unchanged. A hard clear keeps the exact cutoff: it runs at the compaction threshold, where freeing space matters more than the cache. - Discovered tools depend on the provider profile. With the local
search_toolsfallback, pydantic-ai adds a discovered tool's definition to thetoolsarray on providers whose profile has notool_addition_mode(Gemini, Chat Completions, OpenAI-compatible endpoints), which invalidates the whole prefix once per discovery. First-party OpenAI Responses, including ChatGPT, useswith_definitionsand appends the definition after the history instead. Closing the gap for the other providers needs an upstream wire format, or sending every deferred schema up front.
Compaction summaries
A compaction summary is written by repeating the chat's latest request with one
instruction added at the end of its last message, so the provider serves the
whole history from cache instead of a separate model re-reading a flattened
transcript at full price (summarize_on_cached_prefix() in
core/context_compressor.py).
- The exact request.
RequestRecorderCapability(core/prompt_cache.py, registered last) wraps the model that actually sends, and records each chat's latest request: model, settings, parameters, and the messages after pydantic-ai prepared them (framing, tool-search exchanges, the system voice announcement). Repeating the stored history instead would differ from the wire and miss the cache. It also records the agent history the request was prepared from, and the summary is only written that way when that history covers everything being folded (compared byhistory_fingerprint(), which ignores tool output so trimming between turns does not break the match). - Nothing else changes. Tools stay declared and
tool_choicestays as it was, because changing either invalidates Anthropic's cached messages; the instruction tells the model not to call tools, and a reply that does is retried once with a correction. - Fallback. No recorded request (first request after a restart), a history
over 95% of the window, a provider error or an unusable reply all fall back to
the extraction model on a flattened transcript.
compaction_reuse_prompt_cache: falseturns the cached path off. - Thresholds.
compaction_trigger_tokens()andprecompaction_trigger_tokens()take the smaller of a share of the window and a fixed token count: by default 80% or 500k for compaction, 50% or 350k for preparing a summary. The fixed count matters on 1M windows: published multi-needle retrieval results (MRCR v2) for the strongest models stay around 80% up to about 500k and fall to the 70s or lower toward 1M, while mid-tier models fall off near 200k. - Prepared in the background.
schedule_precompaction()starts the next summary while the cache is warm, from two places:ChatProcessorwhen a turn finishes, and the history processor before each request of a run that is past the precompaction threshold but under the trigger, whenprefix_cache_warm()says the chat's last request is likely still cached. The summary is held in_PRECOMPACTIONS, not applied. When the trigger is reached, the history processor applies it if its fingerprint still matches, waits for it if it is still being written, and writes a fresh one if it no longer matches or would leave the context over the trigger. A summary that still covers the history, or one that failed, is not rewritten until the uncovered tail passes 20% of the compaction trigger (PRECOMPACTION_REFRESH_TAIL_SHARE). - Kept across restarts. A finished summary is written to the
prepared_summariestable (chat id,end_index, fingerprint, text) and deleted when it is applied or superseded. The first time the history processor sees a chat past the precompaction threshold in a process,restore_precompaction()loads it back. With no request recorded since the restart, the cold-cache check measures idle time from the last reply's timestamp. - Applied early on a cold cache. Each
RequestSnapshotcarries the Suzent model id and the time it was sent. Before the trigger is reached,cold_cache_compaction_due()applies a finished prepared summary when the history is at or over the precompaction threshold and the time since the chat's last request exceeds the cache TTL estimate: the next request would pay full input price for the whole history anyway. Providers with no TTL estimate (local models) never qualify. - Choosing the writer.
ContextCompressor._prefer_cached_prefix()keeps the cached path while the cache is likely warm. Once it has likely expired, it prices both options withestimate_cost_usd()(genai-prices, the same source as the cost ledger): the conversation's model on the whole prepared request billed as a cache write, against the extraction model on the folded messages, each with a typical summary's output. The cheaper one writes it; if either model is unpriced, the cached path is kept. The choice and both estimates are logged at INFO.
Events and notices
Every pass emits an auto_compaction bus event (start, then complete,
skipped or error). A completed one carries reason: prepared when a
background summary was applied at the trigger, cold_cache when it was applied
early (with idle_seconds), or fresh when it was written on the spot, plus
the summary text. The history processor and /compact persist the terminal
notice with those fields in its metadata, which the chat renders as a divider
(CompactionDivider) and the context panel reads for its "last compacted"
line. schedule_precompaction() emits a precompaction event (start, then
ready or failed, with covered_messages) so the panel can show the
summary being prepared.
Cache TTL estimate
No provider reports how long it keeps a cached prefix, so
core/cache_ttl.py estimates it per model:
- Anthropic uses the TTL the request asked for: 5 minutes, or an hour with
anthropic_prompt_cache_ttl: "1h", whichanthropic_cache_settings()sets on every breakpoint (a longer-lived breakpoint may not follow a shorter one). - Other providers start from
PROVIDER_DEFAULT_TTL_SECONDS: OpenAI, ChatGPT and Gemini 300s, DeepSeek 3600s. Anything else is unknown. - Observed hits raise the estimate. The recording wrapper reports every
response to
observe_request(); a request that read at least 1,024 cached tokens after the model had been idle (in any chat) for 60s or more proves the cache lives at least that long. The longest such gap per model is kept in<data dir>/prompt_cache_ttl.json. Misses never lower it, since a changed prefix misses too. The cost ledger is not used for this: it stores one row per turn, summing every request in it, so it cannot tell whether the first request after a pause hit the cache.
Forked sub-agents
A sub-agent spawned with inherit_context=True starts from its parent's
history, so it can read the parent's cached prefix, but only if the tools and
system prompt in front of that history are byte-identical. A sub-agent built
the ordinary way gets its own system prompt and a narrowed tool list, which
misses the cache on its first request. A fork is built to match instead
(fork_turn_config() in core/subagent_runner.py):
- Same config.
ChatProcessorrecords the config each chat's current turn built its agent from (record_turn_config()). The fork copies its parent's, so it resolves to the same cached agent and the same tool definitions, and overrides only keys that change neither: permission mode, interaction profile, memory, working directory and grants. - Same instructions.
ForkPrefixCapabilitysnapshots the instruction parts of every request. At spawn the parent's latest snapshot is pinned to the fork, and every request the fork makes, follow-ups included, sends those parts instead of its own. Recomputing them would differ in the chat id, permission mode text and memory pin. - Role as a reminder. The sub-agent role, its tool set and the rules that
outrank the inherited prompt (no spawning, no questions to the user) go in a
hidden system reminder with the task (
SUBAGENT_FORK_REMINDER), after the cached history. - Tools refused at call time. The fork keeps every tool definition, and
ToolDenyCapability.before_tool_validatefails a call outsidecall_allowed_tools. Builtins and MCP tools stay callable, as they are for an ordinary sub-agent. Without an explicit tool list the allowlist is the parent's tools, minus the agent tools, so a fork cannot spawn sub-agents. - Same routing key.
prompt_cache_chat_idmakesPromptCacheRoutingCapabilitykey the fork by its parent's chat, so OpenAI and ChatGPT route both to the machine holding the prefix.
The fork falls back to an ordinary sub-agent when the parent has no turn recorded in this process (after a restart) or the fork runs another model. A pinned fork logs its tool-definition drift from the parent, if any, and its cache read share for every request.
Set SUZENT_CACHE_DIAGNOSTICS=1 to also send anthropic_cache_diagnostics on
requests to api.anthropic.com, so a miss is explained by the first point where
the prefix diverged. Anthropic compares a request only with an earlier one that
also opted in, so the parent's requests carry the flag too. It is off by
default because HIPAA-enabled organizations reject it, and other Anthropic
endpoints do not support it. OpenAI's Responses API reports its own
diagnostics without a flag, and the fork log includes either.
For every other chat, ForkPrefixCapability logs two DEBUG lines per model
request. The first says what changed in the prefix since that chat's previous
request: the tool definitions, or the first instruction section whose hash
moved, named by its first line (section 12 '# Memory System' changed (c85e47c9->ccad99b0)). The second gives the cache read share. A miss with
unchanged before it points at the provider (routing, expiry), not at the
prompt. Hashes live in this process only, so the first request after a restart
reads first request in this process.