AI Agent Observability for Loops: Test Repetition Against Progress

A repeated tool call is not enough to call an AI agent stuck. Flag a loop only when a normalized action sequence repeats, the task specific evidence does not change, and neither a declared dependency nor an in policy retry explains the repetition. That rule catches both A A A A and A B A B cycles without mistaking pagination, backoff, or approval waits for failure. The practical minimum is an observation window containing four kinds of evidence: safe action signatures, progress fingerprints, retry policy facts, and dependency leases. Keep the raw trace for diagnosis, but do not let call count or span success become the health verdict. A repeated call is activity, not yet a loop Agent runtimes already expose detailed activity. The OpenAI Agents SDK tracing documentation lists generations, function calls, guardrails, handoffs, and custom spans. The current OpenTelemetry GenAI agent span conventions, marked Development, describe operations for invoking agents and workflows, planning, and executing tools. Those records can establish that an operation ran, how long it took, and whether it returned an error. They cannot define what useful movement means for your task. Four calls to fetch page may collect four different result pages. Four calls to search with the same normalized query and the same accepted evidence may be a loop. The operation name is identical in both examples. Start by deriving two values for every consequential event: Value Example Purpose actionSignature search:topic:timeout Compares the semantic action after volatile fields are removed progressFingerprint brief:sha 4 Records a safe projection of task evidence after the action An action signature should preserve arguments that change the action's meaning and discard values that only create noise. Keep a page token, destination class, normalized query, tool name, and permission scope when they affect behavior. Remove request IDs, wall clock timestamps, random seeds, trace IDs, and reordered JSON keys when they do not. The progress fingerprint answers a separate question: what durable observation would be different if this step helped? For a code agent, it might combine the canonical Git tree digest with the target test result. For a research run, it could cover accepted source IDs and the brief digest. For a publishing job, it could combine the content digest, public resource ID, and verification state. Do not use the model's statement that it made progress. Find the shortest cycle, then test its effect A useful first detector asks whether the complete action window can be described by a short repeating period. The following function checks periods one through three and requires at least two repetitions: For search, search, search, search , it returns period one. For search, read, search, read, search, read , it returns period two. An adjacent duplicate counter catches the first sequence but misses the second. Periodicity alone still does not prove a stall. Evaluate the period beside the progress fingerprints: Order matters. Check reachability before interpreting an action window. Honor a dependency only when it names an owner and has an unexpired deadline. Require complete, safe evidence before calculating a loop. A missing fingerprint is not permission to infer “no progress”; it produces uncertain . I replayed that decision order against six synthetic snapshots at one fixed observation time: Case Sequence or condition Result Catalog pagination Four distinct page signatures; fingerprint grows from 50 to 182 rows working Provider backoff Three identical calls; retryable error; delays of 2s then 4s retrying Same call loop Four identical searches; unchanged evidence digest stuck loop , period 1 Search/read oscillation Three repetitions of an A/B pair; unchanged brief digest stuck loop , period 2 Approval wait Repetition plus named owner and future deadline waiting Redacted evidence Repeated writes but no safe progress fingerprint uncertain The artifact command was: It returned six passing assertions, zero failures, and exactly the two intended loop cases. This is a falsifiable unit test for the rule, not evidence that these proportions occur in production. Keep bounded retries, pagination, and waiting out of the loop bucket Retry evidence needs more than a counter. Record the error class, the applicable policy, attempt number, and actual delay. The AWS Step Functions error handling documentation provides a concrete example of this contract: a retrier matches named errors and uses fields including IntervalSeconds , MaxAttempts , and BackoffRate . The transferable lesson is explicit bounds, not a requirement to use Step Functions. In the fixture, three provider calls with rate limit remain retrying because the error is allowed, the retry count is inside policy, and the observed gaps satisfy the expected 2 second then 4 second backoff. The detector calculates a period one cycle, but retry evidence takes precedence. If the error becomes non retryable, the delays collapse, or the attempt budget is exhausted, that exception no longer applies. Pagination needs a different protection. Do not normalize away the cursor or page token if it changes the requested slice. More importantly, verify the effect: the accepted row count or result set digest should change. A tool can cycle through different cursor strings while the destination remains unchanged, so argument variation without a progress delta is weak evidence. Waiting requires a lease rather than a free text status: Once the deadline passes, reclassify from the remaining evidence. Limit renewals and retain the owner. Otherwise, an agent can hide a loop behind an indefinitely refreshed “waiting for input” message. Calibrate the observation window around the work The fixture uses at most six action events and searches for periods up to three. Those values make the example inspectable; they are not universal production defaults. Choose the window from the workflow's normal rhythm and the shortest harmful cycle you need to catch. An interactive coding agent can repeat a bad read/edit/test pattern within minutes. A scheduled research agent may revisit a source after an hour for a legitimate reason. Use historical healthy and incident replays to measure detection delay and false positives before attaching alerts. Persist enough evidence to reproduce every transition: Store a keyed digest or a non sensitive projection when the underlying evidence could expose customer data, prompts, secrets, or absolute paths. Version the signature normalizer and fingerprint definition. A changed normalizer can otherwise look like a sudden recovery or incident. Alert on a state transition, not every repeated span. For noisy collectors, require two consecutive windows with the same cycle and unchanged evidence. Do not let that hysteresis delay a deterministic outcome failure or a hard permission loss. Diagnosis also does not authorize intervention. A stuck loop finding may justify preparing one bounded nudge, pausing a verified runaway process, or asking for human review. It does not silently authorize a restart, credential change, deletion, external message, or unlimited retry. After any approved action, demand a new progress fingerprint or verified outcome before declaring recovery. Know what this detector cannot establish A poor fingerprint can turn healthy work into an apparent loop. If it hashes only file names, useful edits to existing files disappear. If it includes timestamps, unchanged work looks fresh. A randomized action signature can produce the opposite failure by hiding a stable semantic cycle behind new IDs or reordered arguments. Some useful work is not monotonic. A research agent may reject a hypothesis and return to an earlier source; a coding agent may revert a broken approach. Preserve milestone transitions or accepted evidence changes instead of assuming every counter must rise. Short periods are also a deliberate limit. A six step cycle will not be detected by a maximum period of three. Increasing the window and period ceiling improves coverage but costs memory, delays the verdict, and raises the chance of coincidental repetition. Calibrate from known incidents, and keep uncertain when the evidence cannot support a safe conclusion. The operational rule remains narrow: repeated activity becomes a loop verdict only when the normalized sequence is periodic, useful evidence stays unchanged, and no valid retry or dependency contract explains it. That is enough to turn a busy trace into an inspectable diagnosis without pretending that every repeated tool call is broken. Sidewisp is currently in private preview. Its public site and article library are live, but production agent health collection, runtime adapters, and recovery are not generally shipped. Sidewisp is intended to add a health layer alongside existing runtimes, not replace them or act as an autonomous fixer. If loop evidence is a failure mode you need to see clearly, you can join early access while treating the current product as a preview rather than deployed monitoring.