Datadog LLM Observability for Bedrock: Prove the Inner Trace

Datadog LLM Observability for Bedrock can explain a captured model call or agent invocation, but a visible trace is not yet a health verdict. For an existing Amazon Bedrock Agent, require five receipts: the intended configuration was prepared, the correct alias or version ran, inner trace events arrived, any action handoff reached a defined state, and the promised result exists at its destination. That distinction matters now. AWS says Amazon Bedrock Agents is now Amazon Bedrock Agents Classic and will no longer be open to new customers starting July 30, 2026; existing customers can continue using it. This guide is therefore an audit for existing Bedrock Agents Classic deployments. It is not a greenfield recommendation, and it does not assume that an AgentCore integration has identical telemetry. Start with the exact Bedrock contract you are tracing The first trap is treating “Bedrock tracing” as one feature. Datadog documents automatic tracing for the Bedrock Runtime methods InvokeModel() and InvokeModelWithResponseStream() . Those spans can carry latency, errors, messages, and token usage for a model call. A Bedrock Agent uses a different runtime operation: InvokeAgent . Datadog's current automatic instrumentation reference says its Python Bedrock Agents integration traces the overall InvokeAgent call by default. To expose intra agent steps, the request must use enableTrace=True . AWS gives the same switch an operational meaning: trace enablement follows the agent's reasoning process, actions, and result. The InvokeAgent response is an event stream that can contain output chunks, trace events, errors, citations, and a return control payload. Seeing only the outer call therefore proves less than seeing orchestration, knowledge base, guardrail, and action group evidence inside it. Evidence layer What it proves What it does not prove Bedrock model span A captured model invocation, duration, errors, and available usage fields Which agent version asked for the call or whether the task finished InvokeAgent root span The application invoked the Bedrock Agent runtime That nested orchestration was captured Nested trace events The selected invocation exposed inner reasoning and action steps That the intended draft was prepared or the external effect exists Completed response chunk Bedrock returned the interaction's final response That the promised deliverable passed acceptance Destination receipt A file, record, message, or other expected effect exists and is valid Why an earlier agent step behaved as it did The useful default is to keep automatic instrumentation for supported calls and add a small, content free health receipt around InvokeAgent . Do not upload prompts, credentials, tool arguments, or full responses just to establish state. Store identifiers, timestamps, booleans, counts, and hashes where they are sufficient. Collect five receipts for every important invocation The audit becomes manageable when each boundary owns one receipt. 1. Prepared configuration receipt AWS distinguishes the working draft from prepared versions and aliases. After changing the working draft, you must prepare it before testing or deployment. AWS also recommends checking the agent's preparedAt value. Record: The deterministic rule is simple: If it fails, classify the run as CONFIG NOT PREPARED . Do not debug a detailed trace as though it represents the intended configuration. 2. Invocation identity receipt Reuse the same Bedrock sessionId only when continuing the same conversation. Keep the selected alias or version beside a one way hash of the session identifier. This binds the Datadog root span, the Bedrock event stream, and the operator's expected scope without retaining user content. A recent trace with the wrong alias is wrong scope evidence, not healthy evidence. 3. Inner trace coverage receipt Set trace enablement on the request when the operational question depends on inner steps: Never print the values in a production diagnostic. The receipt only needs: If the root span exists but trace is disabled or no nested event arrives, return TRACE INCOMPLETE . That is a coverage verdict. It does not prove the agent failed. Likewise, a missing Datadog root span should be TRACE MISSING , not AGENT FAILED . Sampling, exporter configuration, transport, unsupported library versions, or instrumentation order can all remove evidence. Datadog exposes a trace sample rate setting, so absence must preserve uncertainty. 4. Action state receipt An agent may invoke a Lambda backed action group, query a knowledge base, or return control to the calling application. In the return control path, the application receives the predicted action and must submit a result to continue. Record whether: an action error occurred; a return control payload arrived; the application submitted the corresponding action result; a response chunk is still pending. When return control is present and no result has been submitted, the correct state is WAITING FOR ACTION RESULT . It has an owner and a next action. Calling it stuck creates noisy alerts; calling it complete loses work. 5. Outcome receipt Datadog can place custom evaluations beside a trace, and those evaluations are useful for subjective quality or policy checks. Prefer a deterministic verifier when the promise is inspectable: file delivery: expected path, MIME, size, checksum, and schema; database write: record key, version, and required fields; outbound message: provider receipt and destination hash; deployment: target revision plus an independent health check; knowledge answer: required citations plus a domain acceptance rule. The outcome receipt should contain the minimum evidence needed to reproduce the verdict. A response saying “done” is not one of those fields. Run the eight state decision rule The accompanying artifact applies the receipts in a fixed order. Earlier boundaries prevent later evidence from creating a false green: I ran this classifier against eight content free cases. All eight matched their expected verdicts: The fixture deliberately gives outcomeVerified: true to the stale preparation case. It still returns CONFIG NOT PREPARED , because an outcome from the wrong prepared configuration cannot certify the intended release. It also gives a complete Bedrock response to the false complete case; without the destination receipt, the final verdict stays red. Route each verdict to one bounded action The classifier is useful only if its states change the operator's next move. Verdict First action Do not do CONFIG NOT PREPARED Prepare the intended draft, confirm preparedAt , then rerun one canary Diagnose old traces as the new release TRACE MISSING Check supported SDK and tracer versions, initialization order, exporter delivery, and sampling Retry the agent as though absence proved failure TRACE INCOMPLETE Confirm enableTrace=True and that nested events reach the Datadog trace Call the outer span full agent coverage WORKING Wait within the invocation's deadline while nested progress changes Alert on elapsed time alone WAITING FOR ACTION RESULT Route the return control request to its owner with a deadline Restart the agent or mark it stuck ACTION FAILED Inspect the first failed action and its error boundary; retry only if the effect is safe Replay an uncertain side effect blindly FALSE COMPLETE Run the destination verifier and repair the missing result Accept the final response text as delivery HEALTHY Retain the compact receipt and close the run Preserve sensitive payloads “just in case” This order also clarifies incident ownership. Datadog coverage problems belong to instrumentation or transport. A return control wait belongs to the application or human who owns the external decision. A failed action belongs to the tool boundary. A missing deliverable belongs to the outcome verifier. One generic “agent error” alert cannot carry those distinctions. Keep the Datadog and Sidewisp boundaries honest Datadog Agent Observability is the right place to inspect captured traces, span structure, model and tool latency, available token usage, errors, and evaluations. The Bedrock integration documentation provides concrete setup and validation steps, including checking tracer status and debugging transmission problems. The audit above adds a release and outcome boundary; it does not diminish trace evidence. It prevents that evidence from answering a question it was not designed to answer alone. Three limitations remain: 1. The fixture validates decision precedence, not the truthfulness of AWS, Datadog, or destination data. 2. Sampling can intentionally remove traces. A coverage SLO needs a controlled canary or another denominator; production trace absence alone is ambiguous. 3. An LLM judge may help with subjective answer quality, but it should not replace a deterministic destination check for an inspectable effect. For existing Bedrock Agents Classic deployments, the practical finish line is therefore: current prepared version, correct invocation scope, complete inner trace when required, resolved action state, and verified outcome. Anything less should remain working, waiting, uncertain, or failed—not green. Sidewisp is currently in private preview. It is intended to add an agent health layer around existing runtimes, but its production Bedrock and Datadog adapters are not shipped. Join the private preview if this evidence to health workflow matches how you operate agents; keep using Datadog and AWS as their current documentation supports today. Sources Datadog Amazon Bedrock integration Datadog automatic instrumentation for Agent Observability Datadog: Monitor agents built on Amazon Bedrock AWS InvokeAgent API reference AWS: Test and troubleshoot agent behavior