How to Own Your AI Agent's Memory
Memory is the part of an AI agent that is genuinely yours, in the sense that it could not have come from anywhere else. The model is a commodity — impressive, interchangeable, and someone else's. What your agent knows about your projects, your preferences, and the decisions you already made is not.
Which makes it worth asking where that lives, and on whose terms.
Three questions that decide ownership
Whatever agent you use, memory is ownable to the degree that you can answer yes to these:
Can you read all of it? Not a summary, not a settings screen showing the last twenty items. The whole thing, in a format you can open.
Can you change it? Agents form wrong impressions. If the only correction mechanism is telling the agent it was wrong and hoping the correction sticks, you do not have edit access — you have a suggestion box.
Can you take it with you? To another machine, another tool, another decade. If the answer requires the vendor to still exist and still offer an export endpoint, that is a conditional yes, which is a no.
Most hosted agent memory fails at least two of these, usually the second and third. Not maliciously — opaque memory is just the path of least resistance when you are building a product and memory is an implementation detail.
What ownable memory looks like
The design that satisfies all three is unglamorous: plain text files on a filesystem you control.
Suzent's memory is Markdown. After an exchange, it picks out what is worth keeping — a preference, a project detail, a fact — and appends it to that day's log. Overnight, a consolidation pass folds those logs into topic pages, merging duplicates and resolving contradictions.
The consequences of that being files rather than rows are practical:
- You can read it. Open the directory. It is a folder of Markdown.
- You can edit it. Fix a wrong fact in your editor. Delete the page about the project you abandoned. The agent's next read sees your version.
- You can version it. Put the memory directory under Git and you have full history, diffs of what your agent learned this week, and the ability to revert a bad consolidation pass.
- You can move it. Copy the folder. That is the migration.
- You can grep it. Every tool you already own works on it.
There is a semantic index for retrieval, because search over a year of Markdown needs one. The important architectural commitment is the direction of authority: the index serves the files, and can be rebuilt from them. Delete the index and nothing is lost. Delete the files and the index is meaningless. That ordering is what keeps the memory yours rather than the database's.
The memory documentation covers the capture and consolidation mechanics in detail.
The parts that should not be portable
One caveat worth stating, because "own your memory" invites an obvious mistake: not everything the agent holds should travel with it.
API keys, provider credentials, device identity, and the local secret store are deliberately excluded from portable state. When Suzent syncs to a private Git repository, the payload builder rejects those paths before push. Memory and skills move; secrets stay on the device.
This matters because the failure mode of naive portability is a memory folder that quietly accumulates credentials and then gets pushed somewhere. Ownership means being able to move the agent without moving your keys.
Why this is the first sovereignty question
Of the five questions in the sovereignty test, memory is first, because it is the one that decides whether the others can even be asked.
If memory is portable and readable, swapping models is a configuration change — identity lives in the files, not the weights. If memory is portable and readable, a provider shutting down costs you an API key. If it is not, then every other form of independence is theoretical: you can leave any time you like, as long as you are willing to start over.
Start with the memory. The rest follows from it.