Log AI tool calls as privileged operations—not just chat transcripts

Source facts: a tool invocation is an execution boundary

Microsoft’s runtime-risk guidance for AI agents describes tool invocations as high-value, high-risk events. In the Microsoft product pattern discussed, context about a planned invocation can be evaluated before execution so policy can allow or block it. The larger principle is product independent: once a model can call an API, update a record, send a message, or run code, observability must extend beyond generated text.

OpenTelemetry publishes GenAI semantic attributes for operations, agents, conversations, retrieval, tool definitions, tool-call identifiers, arguments, and results. Its documentation also warns that messages, queries, arguments, and results may contain sensitive information. Some GenAI conventions are still in development or have moved between repositories, so they are useful building blocks rather than a finished compliance record that every implementation can adopt unchanged.

A transcript can show what the user asked and what the assistant claimed it did. It cannot by itself prove which agent version ran, what context was retrieved, which identity obtained a token, what permission was evaluated, whether approval occurred, which exact parameters reached the tool, what the downstream system accepted, or whether a retry created duplicate effects. Those facts require events from the orchestrator, identity layer, policy point, tool gateway, and target system.

DSE recommendation: create one reconstructable chain from request to effect

Design the audit path before production. Start with the decisions investigators, administrators, customers, and reviewers would need to reproduce after an unexpected change.

Assign stable identities and versions. Record the human requester, agent identity, orchestrator, model and version, governing prompt or policy version, tool server, tool definition version, deployment environment, and target tenant. Distinguish acting as the user from acting as the agent’s own service identity.

Correlate the execution. Generate a request ID and carry it through planning, retrieval, policy evaluation, token issuance, approval, each tool attempt, target-system response, retry, rollback, and final user message. Keep parent and child relationships when one request creates several calls or sub-agents.

Record the control decisions. Capture requested operation, target, authorization scope, policy result, approval requirement, reviewer, decision time, expiry, denial reason, rate limit, and stop condition. Record the enforced decision, not merely the model’s explanation of why an action should be allowed.

Protect content selectively. Prefer hashes, object IDs, field names, counts, classifications, and redacted summaries when full arguments or results are unnecessary. Never copy access tokens, passwords, private keys, or whole sensitive records into ordinary traces. Place any content-rich forensic record behind stricter access and retention.

Keep evidence outside the agent’s control. Send security-relevant events to a protected collector the agent and tool cannot alter or delete. Monitor missing exporters, sequence gaps, clock problems, schema changes, disabled instrumentation, and unexplained differences between orchestrator and target-system logs.

Test reconstruction and response. Run a controlled action, then ask an independent reviewer to identify who requested it, what evidence was used, which permissions and approvals applied, what changed, whether retries occurred, and how to revoke the agent. Exercise unexpected denial, partial completion, tool timeout, and rollback.

Factual boundary: OpenTelemetry conventions standardize useful fields but do not guarantee that an implementation records every event, protects the record, or preserves semantic truth. Logging tool inputs and results can itself create privacy and credential risk. Microsoft product behavior described in the primary source should not be represented as a capability of every agent platform.

Measure tool calls without correlation, unknown agent versions, approval-to-execution delay, denied and retried calls, missing target confirmations, sensitive-field redactions, and reconstruction test success. The goal is evidence that survives the conversation: enough to explain and contain the real operation without turning the logging system into a second sensitive-data store.

Official references

Microsoft Security, From runtime risk to real-time defense: Securing AI agents, January 23, 2026.

OpenTelemetry, Generative AI semantic attributes; stability notices should be reviewed before adoption.

NIST, Generative Artificial Intelligence Profile, NIST AI 600-1.