Multi-Agent Observability: Audit the Coordination Topology

Multi agent observability should answer a stricter question than “did every recorded span finish?” It should tell you whether the agents that actually participated, and the routes they actually used, match the coordination design approved for this run. The practical default is a versioned topology contract : a small manifest of allowed agents, allowed directed edges, and the edges expected in the current run phase. Join that manifest to content free interaction receipts. A successful trace can then be classified as working, waiting, incomplete, unsafe, ambiguous, or false complete instead of becoming green by default. This matters because a trace records what happened. It cannot contain a span for a required delegation that never occurred. It also cannot decide that an observed direct route was forbidden unless you supply the intended graph. The same distinction appears in current architecture guidance: Microsoft’s multi agent reference architecture calls out inter agent message flows and coordination patterns as special observability signals, while the Azure Architecture Center warns that multi agent orchestration adds coordination overhead and new failure modes. Use the lowest complexity that reliably satisfies the task; when multiple agents are justified, make their topology testable. A trace cannot prove the intended topology OpenTelemetry’s tracing API provides the right correlation primitives: trace and span identity, parentage, links, events, timestamps, attributes, and status. Those primitives can describe an observed call tree or asynchronous relationship. They do not declare which agents were permitted, which route version was active, or which edge should have appeared but did not. Suppose an orchestrator delegates research, the researcher hands evidence to a verifier, and the verifier returns a verdict. Every observed event can have status: "ok" in at least five bad situations: the run used yesterday’s routing policy; the researcher called a publisher directly, bypassing review; an unregistered agent entered the graph; the orchestrator delegated the same owned route twice; the parent declared completion before the verifier returned. An “all events OK” query sees no error in those records. A topology audit compares two sets instead: Keep this layer content free. A receipt needs stable run and agent identities, the route kind, topology version, event identity, observation time, and local status. It does not need prompts, responses, secrets, tool arguments, or absolute file paths. The contract is deliberately separate from a fan out completion quorum. A quorum asks whether required branches returned. A wait for graph asks which dependency is blocking progress. A durable handoff receipt asks whether responsibility survived a queue or restart boundary. Topology conformance asks a prior question: is this even the coordination graph we intended to run? Build a versioned coordination contract Start with explicit identities and directed edges. Do not infer the allowed graph from whatever appeared in the latest trace; that merely blesses drift after the fact. The allowed set is not the same as the expected set. A research only phase might expect two delegations and no publisher edge. A complete publication phase might expect the research handoff, verifier return, publisher delegation, and publisher return. Pin that phase specific set when the run begins. Otherwise, an optional edge can quietly become mandatory midway through a failure, or a required edge can disappear from the definition before anyone notices. A compact classifier can use this precedence: 1. stale contract — the event version differs from the pinned version; 2. unknown agent — either endpoint is outside the approved identity set; 3. forbidden edge — the directed route and interaction kind are not allowed; 4. ambiguous route — the same owned edge appears more than once without an explicit multiplicity rule; 5. false complete — a terminal parent lacks an expected edge or verified outcome receipt; 6. waiting — an expected edge is absent, the named dependency is explicit, and the deadline has not passed; 7. incomplete — an expected edge is still absent after its deadline; 8. healthy or working — the observed set matches the current plan, with “healthy” reserved for a verified final outcome. Ordering is important. If a shadow agent uses a forbidden route and the parent is also late, “incomplete” is too weak: the operator first needs to contain an unapproved topology. Conversely, a declared wait before its deadline is not a stall. It is a healthy dependency state that should reach the correct owner without triggering a destructive reset. Dynamic routing is the main limitation. A system may legitimately choose among specialist agents at runtime. Represent that choice as a bounded edge class or produce the exact run plan before dispatch. A wildcard such as orchestrator is easy to maintain but removes most of the diagnostic value. Version changes should be auditable, and a run should never silently adopt a new version halfway through. Sampling is another boundary. Heavy traces may be sampled, but the compact topology receipts used for health decisions cannot disappear under the same policy. If a required receipt is unavailable, report uncertain or incomplete ; do not reconstruct green from a partial trace. Replay drift before you trust completion I replayed nine content free cases against the contract above. The fixture included a healthy completion, current work, a legitimate wait, a missed edge after deadline, a stale topology, a forbidden direct route, an unknown agent, duplicate route ownership, and a terminal parent without a return receipt. The deterministic audit matched all nine expected states. A naive rule—at least one event exists, every local event status is ok , and the parent is not failed—marked all nine cases green . Only one was healthy. Six of those nine naive greens were unsafe, incomplete, stale, ambiguous, or false complete; the remaining two were working and waiting, states that should not be collapsed into completed health. Case All recorded events OK? Topology verdict Operator meaning : Complete graph plus outcome receipt Yes healthy The planned graph and final result are verified Current planned edges Yes working Useful work may continue; do not intervene Missing return before deadline Yes waiting Notify or observe the named dependency Same return missing after deadline Yes incomplete Investigate the first absent expected edge Old topology version Yes stale contract Stop comparing the run with the wrong design Unapproved direct route Yes forbidden edge Contain the route before retrying work Unknown participant Yes unknown agent Verify identity and authority Duplicate owned route Yes ambiguous route Reconcile ownership and possible duplicate effects Parent terminal, return absent Yes false complete Reopen the run; completion lacks required evidence You can reproduce the decision with a small function over normalized edge keys: Run the topology checks before progress, quality, or outcome scoring. Then keep the verdict boundaries explicit: topology conformance proves only that the approved coordination shape was observed; working requires fresh evidence of useful movement, not merely more events; waiting requires a named dependency and deadline; healthy completion requires a deterministic destination or deliverable receipt when one is available; uncertain evidence must remain uncertain; any active recovery needs bounded authority, visibility, and post action verification. This gives an operator a narrow adoption rule: do not trust a multi agent completion until the run’s pinned topology, current phase, and final outcome receipt agree. A matching graph is necessary evidence, not proof that the answer is correct. Sidewisp is currently in private preview. Its public early access site and interactive demonstration are live, but a production multi agent monitoring adapter, live health collector, and automated recovery executor are not shipped. Sidewisp’s intended role is to add a health layer around existing runtimes and make evidence, severity, uncertainty, and the safest next action easier to inspect—not to replace the runtime or act without human authority.