← Frank Zhang

Engineering Notes · Agent Systems

Agent Memory Is Easy to Store. Trustworthy Recall Is Harder.

Drafted

An agent can save a sentence in a database. The harder question is whether a later session should use it—and what should happen before that sentence is saved at all.

Agent Memory Bridge (AMB) is my open-source project for carrying project knowledge across agent sessions. It grew from a practical problem: decisions and their reasons get lost in conversations, but blindly replaying old context can bring back stale assumptions, unverified fixes, or sensitive material.

Remember the reason, not the whole conversation

AMB's local-first model gives an agent a way to record an explicit decision and its context, then recall relevant knowledge in a later session. Its Model Context Protocol (MCP) interface exposes memory operations to compatible clients. The store keeps revision relationships so a corrected decision can supersede an older one rather than quietly erasing its history.

That does not make a stored claim true. Provenance helps a reader find where a claim came from; it is not proof of correctness or authenticated identity. A later agent still needs to compare recalled guidance with current code and evidence. Memory explains why a choice was made; the live repository shows what exists now.

Recall is a starting point for a check, not a substitute for one.

For example, a team might record why a project uses one authoritative store. A later session can retrieve that reason before changing storage—but must still inspect the current implementation and ask whether the original constraints remain valid.

Convenience creates a new boundary

Explicit memory has a cost: someone or something must decide what is worth retaining. In research issue #72, I investigated whether lower-friction capture could improve that experience without silently promoting unverified material into trusted guidance.

There are several different steps: observing a source, extracting a candidate, storing it, reviewing it, and promoting it to trusted guidance. They are not interchangeable. AMB supports bounded, structured capture into a review lane; that is not the same as automatically extracting useful knowledge from ordinary conversations. A candidate can be stored even when it is not returned by ordinary recall.

The distinction matters because review controls authority, not confidentiality. If sensitive material is persisted before review, keeping it out of recall does not undo the write.

The check that failed

The published #72 report describes an isolated synthetic privacy check. A known key-pattern input was rejected before storage, but an opaque synthetic identifier designated sensitive by the fixture reached a hidden review candidate. The no-storage check failed. No real secret or production database was used; this was not evidence of a production leak.

It was also not a solved security issue. A pattern match can catch known shapes without recognizing every sensitive value. A caller-provided safety label is not proof of confidentiality. Other passing tests in the report did not erase this negative result or establish protection against arbitrary sensitive content.

The published material includes a redacted evidence capsule; it does not expose the complete raw run package. The result is historical evidence tied to the research snapshot, not a claim about every version or deployment.

What I chose not to build

I deferred broader automatic capture rather than treating a review queue as a privacy guarantee. The follow-up #73 design research examines approved sources, minimization before writing, retention surfaces, and the limits of detecting opaque sensitive data. Its closeout does not authorize an extractor or automatic persistence; proposed synthetic experiments remained not run.

That decision trades convenience for a more explicit boundary. Users must still identify valuable knowledge and request recall. It also leaves difficult questions open: which sources are safe to inspect, which host or provider may retain a copy, and whether the benefit justifies the operational and review cost. A future experiment would need separate authorization and evidence across those boundaries.

What the evidence does—and does not—show

The project provides a concrete path for storing and recalling explicit project knowledge, with revisions and governance. The research also shows why a successful candidate or a passing known-pattern test is not enough to claim safe automatic learning.

A small synthetic comparison found that a standing recall procedure mattered in the tested sessions. Its sample and host scope were limited; it did not prove broad productivity gains or that packaging alone caused the result. The engineering lesson is narrower: make the boundary visible, test a plausible failure, and preserve the failure when it changes the design.