Pydantic AI Agent Memory: Audit History Before Replay

Pydantic AI agent memory is replay ready only when six independent checks pass: the messages round trip through the supported serializer, the history came from an authorized server side source, the required system prompt survived, tool calls and results remain operationally honest, every message belongs to the intended conversation, and the application has a receipt for the expected outcome. A valid JSON payload proves only the first check. That distinction matters because Pydantic AI deliberately repairs some provider invalid histories. A cancelled tool call can become a provider valid interrupted return. An orphaned tool result can be removed. Those repairs help the next model request succeed, but they do not prove the abandoned tool finished or that the user's deliverable exists. Treat serialization as the first gate, not the verdict Pydantic's official messages and chat history documentation recommends ModelMessagesTypeAdapter for storing and loading ModelMessage history. It preserves the message fields that fit the adapter's schema, while a JSON round trip normalizes values that have no native JSON representation. That is the correct persistence boundary. It is not an application health check. To make the difference measurable, I ran seven content free histories through Pydantic AI 2.21.0. Each history was serialized with ModelMessagesTypeAdapter.dump json , reloaded with validate json , normalized, and hashed. The cases then went through separate checks for prompt continuity, tool pairing, conversation scope, trust, and an application supplied outcome receipt. All seven normalized JSON round trips matched. Only the control case was ready to replay: Case JSON round trip Operational verdict Complete history plus outcome receipt Equal READY Missing system prompt Equal SYSTEM PROMPT GAP Interrupted tool call Equal INTERRUPTED TOOL Orphaned tool result Equal ORPHAN RESULT REMOVED Mixed conversation IDs Equal SCOPE DRIFT Model response without outcome receipt Equal MISSING OUTCOME RECEIPT Client supplied history Equal UNTRUSTED HISTORY The result is not an argument against the adapter. It shows why a schema valid archive and a healthy agent state are different claims. The reasonable default is to persist the exact adapter output, retain a stable conversation identifier, and keep a separate outcome record. Do not flatten the messages into ad hoc role/content pairs if you need tool metadata, run boundaries, system prompts, or application annotations to survive. A minimal persistence path looks like this: After loading, run the other gates before passing the result to message history . Audit prompt continuity and conversation scope Pydantic AI's history contract contains a subtle default: when message history is non empty, the framework assumes that the history already contains a system prompt. It does not generate a new one for that run. The documentation points to ReinjectSystemPrompt when a database, frontend, or compaction path does not round trip the prompt. That means “the old user messages are present” is insufficient. A persistence job can retain every chat turn and still remove the instruction that defined the agent's authority or output contract. Record the prompt contract as a non secret identifier, not as a copy of sensitive instructions in a health stream. For example: At replay time, verify that the loaded history contains the prompt shape required by that revision. If your application intentionally uses dynamic instructions instead of persisted system prompts, check that contract explicitly rather than treating absence as automatically unhealthy. Conversation identity needs its own test. Current Pydantic AI messages can carry both run id and conversation id . A new run should receive a new run ID, while the conversation ID correlates turns. The source for version 2.21.0 documents and implements separate resolution rules for the two identifiers. A practical replay gate should reject or quarantine a history when: more than one non null conversation ID appears without an explicit merge decision; the requested tenant or user does not own the conversation; the history was loaded under one authorization context and replayed under another; a caller attempts to reuse a prior run ID as if it were the conversation key; a fork was intended, but the application retained the original conversation identity. These are scope decisions. They cannot be recovered from message text safely, and a model should not adjudicate them. Inspect repaired tool history without calling it complete Model providers generally reject a tool result without a matching call, or a call whose required result never appears. Pydantic AI's current history cleaning behavior makes regular, locally executed tool history provider valid before a request. The version pinned 2.21.0 source shows the order: 1. remove orphaned regular tool results; 2. synthesize interrupted results for dangling regular tool calls when repair is appropriate; 3. merge compatible consecutive messages after pairing is valid. Synthesized returns are marked in metadata with pydantic ai synthesized tool return and use the neutral interrupted outcome. In the controlled replay, the interrupted case gained one marked return. The orphan case lost its unmatched return. Both resulting histories were easier for a provider to accept. Neither became evidence that the tool's external effect happened. Use the marker as an incident signal: Do not automatically retry every interrupted call. A timeout can occur after an irreversible external effect but before its result reaches history. The bounded next step is to query the destination with a safe idempotency key or business identifier. Retry only when the destination proves the effect is absent and the operation is safe to repeat. Orphan removal also deserves a receipt. If a loaded history contained a result that cleanup later removed, preserve a content free audit event with the conversation ID, hashed tool call ID, observed time, and repair class. Do not preserve raw tool arguments or results unless the incident genuinely requires them. The experiment used Pydantic AI's private clean message history symbol to reproduce the documented pipeline exactly. That import is appropriate for a pinned diagnostic fixture, not for application code. Private APIs can change without compatibility guarantees. Production checks should use supported message types, documented metadata, application receipts, and tests pinned to the version being deployed. Keep client history outside the authority boundary Pydantic's documentation is explicit about client supplied history: server side agent surfaces are stateless, so a client that can submit history can fabricate tool calls, tool results, system prompts, or approvals. sanitize messages narrows several unsafe shapes, but sanitization does not establish that the submitted history is true. Treat possession of a JSON transcript and authority to resume work as separate facts. The server should authenticate the caller, authorize the conversation, construct the permitted toolset from server side identity, and revalidate high stakes effects inside the tool function. If a paused run matters, persist it on the server and resume it from server owned state. Do not accept a browser's claim that an approval happened merely because an approval shaped message validates. This is a documented trust boundary, not a vulnerability report. The operational failure is an application treating an untrusted history as an authority record. A compact precedence rule prevents a plausible lower level check from hiding a more important failure: The order is deliberate. There is no value diagnosing a missing deliverable from a history the caller was never allowed to resume. Require an outcome receipt after history passes The final gate belongs to the application, not the model history framework. Define the expected effect before the run. For a coding agent, it might be a commit containing a specific change plus passing tests. For a support agent, it might be a ticket update visible through the ticket API. For a scheduled report, it might be an immutable object at the expected destination with the correct reporting interval. Store a content minimized receipt: The receipt should come from the strongest available deterministic readback. A model saying “done” is activity evidence. A tool return saying “accepted” is transport evidence. A fresh destination read showing the intended revision is outcome evidence. This also handles legitimate waiting. If the agent is paused for human approval, the correct state is not MISSING OUTCOME RECEIPT ; it is a waiting record with an owner, decision needed, deadline, and resume token. Only classify the run as stuck when the expected progress window closes without a valid wait or outcome. The seven case replay has a narrow limit: it tests selected message history failure states under Pydantic AI 2.21.0, not every provider, builtin tool, UI adapter, or long term memory product. Rerun the fixture when upgrading the framework, changing serialization, adding a frontend adapter, or altering tool execution. Sidewisp's intended health model includes context persistence, tool reachability, useful progress, and outcome verification. Sidewisp is currently in private preview. Production agent health collection and a Pydantic AI adapter are not generally shipped, so this guide is an independent operating rule rather than a claim that Sidewisp already performs the audit. The replay decision is therefore straightforward: use ModelMessagesTypeAdapter for durability, then require authority, prompt continuity, honest tool pairing, conversation scope, and an external outcome receipt. Provider valid history is useful evidence. It is not the same thing as a healthy agent.