Enforce approval outside the model before an AI agent commits a consequential action

Source facts: the model cannot be its own escalation authority

Microsoft’s defense-in-depth guidance for autonomous AI agents describes deterministic human-in-the-loop review as an application-layer control. It warns against letting the model decide when review is needed because probabilistic reasoning, ambiguous instructions, or adversarial input can bypass the escalation mechanism. The guidance recommends defining triggers in code and enforcing them through the application or orchestrator, including during tool execution.

Microsoft’s related Zero Trust guidance says users should be able to set boundaries for what agents access, do, and remember; high-risk or irreversible actions should require approval; and organizations need reliable system-level methods to pause or stop agents. The design point is separation of duties. The model may propose and explain an action, but a deterministic policy point decides whether execution is allowed, denied, or held for a qualified person.

Human approval is not automatically safe. A reviewer can be rushed, receive a vague description, miss several individually small actions that create a large combined effect, or approve a request whose parameters change before execution. Approval must therefore bind to the proposed operation and provide enough context for an informed decision.

DSE recommendation: make approval a narrow execution capability

Start with business consequences rather than a generic “confirm” button. Define which decisions remain human-owned and what evidence a reviewer needs before accepting them.

Classify actions. Inventory every tool operation and rate data sensitivity, people affected, financial value, external visibility, privilege change, reversibility, blast radius, legal or safety consequence, and ease of independent verification. Define actions that are prohibited, autonomous within limits, or approval required.

Encode mandatory triggers. Put rules in the orchestrator, policy engine, or tool gateway. Require approval for defined recipients, data classes, access changes, production deployments, deletions, payments, external sharing, credential operations, and threshold crossings. The model must not be able to suppress or rewrite those rules.

Present the real effect. Show the tool, authenticated identity, exact target, before-and-after values, recipients, records affected, data to be disclosed, source evidence, dependencies, reversibility, and expected follow-on actions. Distinguish model-generated rationale from verified system facts.

Bind and expire the decision. Issue approval for the specific action hash, parameters, identity, environment, and short execution window. Any changed recipient, scope, amount, file, query, or tool definition should require a new decision. Prevent replay and record denial, timeout, cancellation, and execution status.

Evaluate compound behavior. Aggregate related steps before review. A sequence of read, export, upload, and share operations may be consequential even if each isolated tool call is below a threshold. Limit recursion, parallelism, total records, cumulative value, and repeated approval prompts.

Protect the reviewer. Route decisions to a role with authority and context, avoid self-approval, pace notifications to reduce consent fatigue, provide a safe deny-and-investigate path, and never punish a reviewer for stopping an ambiguous action. Use dual approval where the business risk justifies it.

Exercise containment. Test forged approval, stale approval, altered parameters, unavailable reviewer, prompt injection, incremental escalation, partial completion, rollback failure, and emergency stop. Verify that disabling the agent identity and tool gateway actually blocks pending work.

Factual boundary: Human approval reduces risk but cannot prove an action is correct or harmless. The threshold is environment specific; no source supplies a universal list of consequential actions. Deterministic enforcement should not be confused with a guarantee that the policy itself is complete.

Measure approvals without sufficient context, altered requests, repeated prompts, reviewer turnaround, denied high-risk actions, expired approvals, self-approval attempts, and rollback success. The mature outcome is not a human clicking after the model. It is an application that refuses to create consequential authority until a qualified person approves the exact effect.

Official references

Microsoft Security, Defense in depth for autonomous AI agents, May 14, 2026.

Microsoft Learn, Reduce autonomous agentic AI risk.

NIST AI Resource Center, AI RMF Playbook.