New Relic LLM Observability: Prove One Route Owns Each Call

New Relic LLM observability can show model latency, tokens, errors, traces, and AI response data. It cannot, by itself, tell an operator whether two collectors counted the same model call or whether the agent produced the requested external result. The practical default is to name one instrumentation owner for each model call boundary , correlate every record to a content free call ID, and keep a separate receipt for the deliverable. That rule matters because New Relic documents several legitimate paths. Its native AI monitoring uses APM agents. New Relic also documents OpenLIT over OTLP for traces and metrics and OpenLLMetry over OTLP for traces. LiteLLM has a separate New Relic integration built around its callback and the New Relic Python agent. Those paths are options, not evidence that all should instrument the same boundary. A dashboard can be populated while ownership is wrong, evidence is stale, a required field is absent, or a completed model call has no verified outcome. Declare the route before trusting the chart Start with a deployment manifest, not a query. It should identify the service, the boundary being instrumented, the one owner allowed to report that boundary, the signal kinds that owner may emit, the correlation field, and the freshness limit. The distinction between an owner and a signal kind prevents a crude deduplication mistake. One declared owner may deliberately emit an APM span and an AI message event for the same call. Those records are complementary when they share the same call ID and the deployment contract expects both. An OpenLIT record and an OpenLLMetry record for that same boundary are two owners, even if their fields look similar. The manifest should be produced by the deployment that enabled instrumentation. Do not infer it from whichever entity happens to appear in the UI. The setup paths establish identity differently: New Relic AI monitoring begins with an APM agent and supported library or framework. OpenLIT sends traces and metrics to New Relic's OTLP endpoint. OpenLLMetry sends traces to that endpoint, and New Relic derives the service entity from the OpenTelemetry service.name resource attribute. LiteLLM enables a newrelic callback and uses the New Relic Python agent for APM telemetry. Its documentation says the callback logs an initialization message and that trace details may take two to three minutes to appear. This is enough to require an explicit route record. It is not evidence that any particular pair will always duplicate a call. The safe operational claim is narrower: if two owners are observed for a boundary whose manifest permits one, totals and health verdicts are ambiguous until the deployment is reconciled. Use an opaque call identifier rather than a prompt hash. A useful observation envelope can stay content free: Prompt and response content are unnecessary for ownership, freshness, token field presence, or destination verification. LiteLLM documents both a New Relic specific turn off message logging setting and an environment switch that disables AI monitoring content recording. Treat content retention as a separate privacy decision; do not turn it on merely to make the ownership audit work. Replay the eight states that a green view hides The accompanying audit uses eight synthetic calls. It applies this precedence: 1. no observation; 2. expected owner absent; 3. more than one owner; 4. unexpected signal kind or missing required field; 5. stale evidence; 6. legitimate waiting; 7. completion without an outcome receipt; 8. completion with an outcome receipt. Precedence matters. A stale record from the correct owner is not healthy. A fresh record from the wrong owner is not healthy either. Waiting is evaluated only after ownership, shape, and freshness pass, so an approval pause cannot conceal broken collection. The complete fixture contains no prompts or responses. Running it produces eight different verdicts: The classifier is deliberately small: Each non healthy verdict points to a different repair: Verdict What it establishes Bounded next action NO TELEMETRY No record arrived for the expected call Check instrumentation initialization, exporter reachability, and the query window ROUTE DRIFT Data arrived, but not from the declared owner Compare the deployment manifest with the running process and disable the unintended path MULTIPLE OWNERS More than one instrumentation owner observed the boundary Quarantine the call from totals; choose one owner or record a time bounded migration exception SCHEMA GAP Ownership is correct, but evidence is unusable or incomplete Fix the field mapping or signal kind contract before alerting on it STALE The latest evidence exceeds the allowed age Inspect exporter delay, queueing, clock alignment, and query time WAITING Collection is healthy and a named dependency remains Notify the owner or wait until the recorded deadline; do not restart the agent OUTCOME UNVERIFIED The model call completed without proof of the requested effect Run the deterministic destination check HEALTHY Ownership, evidence, work state, and outcome all agree Keep the receipt and apply the normal stability window MULTIPLE OWNERS should not automatically delete or merge records. During a planned migration, dual collection can be useful. Make the exception explicit with a start time, end time, owners, and reconciliation rule. Keep migration observations out of production cost and reliability denominators until the two paths are compared. Otherwise, an apparent token spike may be an instrumentation change rather than a behavior change. The fixture also shows why a generic red state is weak. ROUTE DRIFT is a deployment problem; STALE may be an ingestion or query window problem; WAITING is not a failure; and OUTCOME UNVERIFIED requires a destination check rather than another trace query. Join observability to an outcome receipt New Relic's AI monitoring documentation describes performance, cost, token, response, trace, and user feedback evidence. Those are useful signals about the AI layer. A model response can still be followed by a failed tool call, an uncommitted file, an email that was never sent, or a job that is waiting for approval. For that reason, keep the destination receipt outside the model call telemetry: The destination and verification method depend on the work. Use an object version for a file upload, a commit hash plus checks for a code change, a provider message ID for a delivery, or an API read back for a configuration mutation. A command exit code is weaker when the promised outcome exists elsewhere. In the replay, false complete and healthy have the same two New Relic side signal kinds, owner, fields, and freshness. Only the destination receipt changes. That is the operational boundary: LLM observability explains the model call evidence; the receipt proves that the agent's intended result exists. A practical rollout is small: 1. Pick one real model call boundary. 2. Record its expected instrumentation owner and allowed signal kinds in the deployment. 3. Generate one synthetic call ID and query every relevant New Relic event type for it. 4. Fail the rollout if the expected owner is absent or any undeclared owner appears. 5. Verify required fields and a bounded freshness interval. 6. Record working , a named waiting dependency, or complete separately. 7. Require a deterministic destination receipt before moving complete to healthy. 8. Repeat after an SDK, callback, exporter, or agent upgrade. This procedure has a limitation: it does not prove that every New Relic supported integration will double instrument every framework combination. It proves whether your observed deployment matches its declared ownership contract. It also does not replace New Relic's compatibility checks or a full OpenTelemetry schema conformance test. Sidewisp's intended role is adjacent to this distinction: combine evidence about reachability, useful progress, tools, outcomes, time, and budget into an agent health view. Sidewisp is currently in private preview. The live product is an early access website and demonstration; a production New Relic adapter, monitoring engine, and automated recovery executor are not shipped. Use the audit above with your current telemetry and destination systems rather than assuming Sidewisp collects or fixes them today. The resolution is simple enough to enforce: one declared instrumentation owner per model call boundary, multiple signal kinds only when the manifest permits them, fresh content free evidence, an explicit waiting state, and an independent outcome receipt. A green New Relic view becomes trustworthy for agent operations only after those boundaries agree.