Agentic AI Observability for Fan-Out: Prove the Completion Quorum

In a fan out agent run, a finished parent is not proof that the work is complete. The parent may have collected the fastest branch, ignored a required review, merged a stale result, or counted the same branch twice. Agentic AI observability needs a completion quorum: a frozen list of expected branches plus fresh, unique, outcome verified receipts for every required branch and the declared number of optional branches. The practical default is strict. Freeze the branch manifest before dispatch, identify required and optional work, set one deadline, and let only verified receipts satisfy the quorum. Before the deadline, missing coverage can be waiting . After the deadline it is incomplete . If the parent reports success without coverage, label it false complete instead of turning a convenient terminal flag green. Traces show the fan out, but they do not define completion The OpenAI Agents SDK tracing documentation records generations, tool calls, guardrails, handoffs, custom events, parent span IDs, and timestamps. The OpenTelemetry GenAI agent span conventions, currently marked Development, describe operations including agent invocation, planning, and tool execution. Those records are useful evidence of activity. They do not know which branches the operator required or what observable result each branch owed. A trace can contain four successful child spans while the unspawned fifth branch—the security review, destination check, or regional data source—never appears. Looking only at emitted spans creates a selection problem: missing work leaves no span to inspect. General batch systems make the expected set explicit. The Kubernetes Job documentation distinguishes parallel work with a fixed completion count from work queue jobs. For Indexed Jobs, completion requires one successful Pod for each index; the documentation also warns that more than one Pod may start for the same index and only the first successful completion counts. Agent fan out is not a Kubernetes Job, but the operational lesson transfers: identity and expected coverage matter, not just a total number of “success” events. For agentic work, the parent should freeze a manifest like this before it delegates: The freeze matters. If the parent can silently remove a slow required branch after dispatch, the denominator changes to fit the observed result. A manifest revision can be legitimate, but it needs a new version, reason, and approval boundary rather than an in place edit. Make each receipt prove an outcome A branch receipt needs more than status: succeeded . Give it the frozen branch identity, observation time, an evidence digest, and a result from an independent outcome check: The digest should cover a canonical, non secret projection of the promised result. For a test branch, that might include the commit, test command, exit status, and normalized test summary. For research, it could cover the selected source URLs, retrieval times, and claim ledger. For a delivery branch, use the destination object identity and a read side verification result. Do not hash raw prompts, credentials, or sensitive payloads merely to make the receipt look rigorous. Evaluate receipts in an order that preserves uncertainty: 1. Reject an invalid manifest, an unknown branch ID, or conflicting duplicate receipts as uncertain . 2. Return incomplete when a required branch explicitly fails. 3. Return unverified when a required success is stale, lacks an evidence digest, or has no independent outcome check. 4. Count coverage only when every required branch has one fresh verified success and optional verified successes meet the declared quorum. 5. Return complete only when coverage is met and the parent is terminal; otherwise return ready to finalize . 6. If coverage is missing, return waiting only while the deadline remains open. 7. Return false complete when the parent terminates early, or incomplete when the deadline expires first. Conflicting duplicates deserve special treatment. Two receipts for the same branch with different evidence digests may indicate a retry, split brain, or non deterministic output. Arbitrarily accepting the newest result hides the conflict. Keep the state uncertain until a branch specific policy identifies the authoritative attempt. Replay seven completion states The accompanying artifact runs one deterministic classifier over seven synthetic fan out cases: The replay produced: research 42 has verified receipts for both required branches and one of two optional branches, exactly meeting its optional quorum. Because its parent is terminal, it is complete . merge 17 has full required coverage but remains ready to finalize because parent execution is still open. The paired failure cases expose why a single “running/completed” field is inadequate. approval 09 is missing one required receipt but remains inside its deadline, so it is waiting . early parent 08 has the same coverage gap but a terminal parent, so it becomes false complete immediately. The classifier does not wait for the deadline to admit that the success claim is unsupported. unchecked 24 includes a successful destination check receipt whose outcomeVerified field is false. It is unverified , not complete. conflict 15 submits two different digests for the same branch and therefore remains uncertain . The artifact treats missing evidence and contradictory evidence as different operational problems. This is a falsifiable decision rule, not a measurement of production failure frequency. Seven constructed cases demonstrate branch coverage and state ordering; they cannot establish universal deadlines or show how often real agents lose fan out work. Calibrate the quorum without making it cosmetic Start with branch semantics. A security review, destructive action approval, or destination verification is usually required even if several optional enrichment branches succeed. Never let a numeric optional quorum override a named required branch. If “two of three sources” is acceptable, encode all three as optional with quorum two and preserve their identities. Set the deadline from the task class, not from a global agent timeout. An interactive code review and an overnight regional crawl have different healthy waits. Record collector time and source time; reject receipts observed before the manifest freeze or implausibly after the observation time. If clocks cannot be trusted, surface uncertain rather than guessing freshness. Persist the manifest and receipts durably enough to survive parent restarts. A reconstructed list based only on currently visible children can omit a branch that failed before telemetry flushed. Store a manifest version, parent run ID, branch identity, receipt attempt identity, and the final classification reason. If duplicate execution is possible, make the authority rule explicit instead of relying on arrival order. Finally, separate diagnosis from intervention. A missing receipt may justify notifying an owner, requesting evidence, or preparing one bounded retry. It does not authorize repeated spend, deletion, credential changes, or silently dropping the branch. After an approved retry, require a new receipt and re run the quorum check. The boundary is important: a completion quorum proves declared coverage, not that the manifest captured every real requirement. A weak branch check can also verify the wrong artifact. Review manifest design and outcome checks as carefully as classifier code. Sidewisp is currently in private preview. Its public site and article library are live, but production agent health collection, runtime adapters, and recovery are not generally shipped. Sidewisp is intended to help make evidence, waiting, false completion, and approval boundaries visible alongside existing runtimes—not to replace the runtime or act as an autonomous fixer. If completion quorums match a failure you need to inspect, you can join early access without treating the preview as a deployed monitoring claim.