AI Agent Observability for Tool Timeouts: Verify the Effect

A tool timeout tells you that the caller stopped waiting. It does not tell you whether the tool performed its side effect. For an AI agent that can send a message, create a ticket, reserve a slot, or charge an account, treating timeout as failed can turn an ordinary network fault into a duplicate real world action. The reasonable default is to freeze blind retries, keep one stable operation identity, and reconcile the intended effect. Accept one of three answers: verified applied, verified not applied, or indeterminate. Only the second answer can enter a retry decision, and even then the provider contract must support a retry with the same identity and unchanged parameters. This is an observability problem because a healthy verdict depends on evidence beyond the tool call span. The agent needs a receipt for what it intended, what transport returned, and what the external system now contains. A timeout leaves three different facts An agent usually records one convenient fact: the tool call raised a timeout. The useful operational record has three layers: 1. Intent — the exact logical operation the agent committed to perform. 2. Transport — whether the caller received a response, rejection, or no answer. 3. Effect — whether the target system contains the intended result, contains no result, or cannot be queried confidently. Those layers can disagree. A request may reach the provider, create the object, and lose the response on the way back. It may fail before dispatch. It may return success while an asynchronous downstream step never produces the promised deliverable. None of those cases is described well by a single success: true false field. Stripe's advanced error handling documentation makes the ambiguity explicit: after a network error, the client does not know whether the server received the request. Its recommended retry path reuses the same idempotency key and the same parameters until the client receives a definitive result. The same page treats a 500 response as indeterminate because a request may still produce a user visible side effect. AWS documents a related boundary in its Durable Execution guidance. At least once replay is safe for idempotent operations; external side effects need at most once handling or a service side idempotency contract. AWS also warns that neither per attempt semantic means “exactly once” for the whole workflow. The practical conclusion is narrower than “add retries.” First decide whether the operation is safe to repeat. A read, an upsert keyed by a stable record ID, and a provider call with a documented idempotency key are different from sending a one shot notification through an API that has no deduplication contract. Record an effect receipt before adding retries An effect receipt is a small local record created before dispatch. It is not the provider's response alone. It correlates the committed intent with later evidence: operation id identifies the logical action across process restarts. request hash prevents an agent from reusing that identity for changed parameters. The idempotency key is separate because not every provider supports one, and providers define different retention windows and replay behavior. The probe describes how the effect was checked; a cached list endpoint is weaker evidence than a direct read by a unique external reference. Do not store secrets, prompt bodies, message content, or full tool arguments in this record. Hash a canonical, redacted intent and retain only the fields needed to reconcile the effect. If the provider accepts customer metadata, attach the stable operation ID there so a later webhook or read can correlate an object even when the original response vanished. The verdict should use explicit evidence states: Verdict Evidence Next action verified applied One matching effect or a trustworthy replayed provider receipt Do not retry; continue to outcome verification verified not applied An authoritative query proves zero matching effects Consult the provider contract before one bounded retry indeterminate The response is missing and no authoritative effect check is available Wait, reconcile, or ask a human; do not invent certainty duplicate effect More than one matching effect exists Stop retries and enter a compensating or human repair path false success Transport returned success but the promised effect is absent Treat the run as unhealthy even though the command completed unsafe retry The same intent was retried under a new key or changed request hash Stop; the deduplication boundary has been broken This table separates activity from useful progress. Another retry is activity. A confirmed single effect is progress. A legitimate reconciliation window is waiting, while repeated fresh keys with no stable receipt is unsafe execution. Run the six case classifier I built a six case NDJSON fixture to test the rule. It includes a lost response with a matching provider receipt, a timeout with unavailable evidence, a provider that creates two effects despite a repeated key, a success response with no resulting object, an authoritative zero effect result, and a changed key retry. The core classifier is deliberately small: The fixture resolves to one case in each state: All six expected assertions pass. The most important result is the second line, not the happy path: a timeout with no authoritative read path remains indeterminate . A fresh retry would make the dashboard busier while making the real world state harder to recover. The fixture also catches a tempting shortcut. A provider receipt is useful only when it binds to the original request hash. A receipt for a different payload cannot prove that the intended effect happened. Likewise, a transport 200 is not outcome verification; the ack without deliverable case is false success because the external object is absent. In production, run reconciliation on a bounded schedule. Query by the stable operation ID or provider idempotency key, record evidence freshness, and stop after a fixed deadline. If the result stays indeterminate, route the decision to someone with authority over the affected system. Do not let a generic “retry up to three times” policy cross a side effect boundary. Where the contract stops An effect receipt reduces ambiguity; it does not create an exactly once guarantee. The provider may expire idempotency keys, ignore them on some endpoints, accept a request before an internal asynchronous failure, or expose a read model that lags behind the write. A probe can also be wrong because of caching, partial permissions, or a non unique lookup. Put those limits beside the verdict: retain the provider's documented idempotency window and scope; use the same key and the same canonical request during a permitted retry; label probe confidence and freshness; distinguish an authoritative zero from “not visible yet”; cap reconciliation time and retry count; require human approval for compensating or irreversible actions; verify the user's intended outcome after the effect is confirmed. This pattern is especially valuable for long running agents because process recovery often loses the transport response while external work continues. Persisting the receipt before dispatch gives a restarted agent a stable place to resume investigation. It should resume from indeterminate , not from “probably failed.” For AI agent observability, the operating rule is simple: a timeout is a transport observation, not an effect verdict. Preserve one operation identity, correlate provider and read side evidence, and refuse to call the run healthy until the intended external result is verified. Sidewisp's product direction includes outcome health, tool failures, retries, evidence, and human approval boundaries. This article describes an operating pattern, not a shipped monitor. Sidewisp is currently in private preview. Production agent health collection, runtime adapters, effect receipt monitoring, and recovery are not generally shipped. The public site and article library are live, and readers can join early access without granting Sidewisp authority over their agents.