How do you govern AI agents in an enterprise?

In detail

The trust problem governance exists to solve

Trust in what AI agents produce has to be earned before an enterprise lets them act. That caution is rational — an agent that acts on its own, on real systems, is a real liability. Governance is the set of controls that lets an organization deploy agents despite that, by making their behavior visible, bounded, and accountable.

Know what you have

You can’t govern agents you can’t see. The first step is an inventory: every agent running in production, what it does, what data and tools it touches, and who owns it. Shadow agents — ones spun up by a team and never registered — are where governance fails silently. The NIST AI Risk Management Framework makes the same point for AI systems generally: one of the outcomes under its GOVERN function is that mechanisms are in place to inventory AI systems, resourced according to the organization’s risk priorities.

Control what they can do

Each agent should have the minimum permissions its job requires, and high-stakes or irreversible actions should route through an approval gate rather than execute autonomously. This is governance as engineering: the limits are enforced by the system, not promised in a policy doc. OWASP’s entry on excessive agency names the three root causes these controls address: excessive functionality, excessive permissions and excessive autonomy. The glossary entries on least privilege and excessive agency go into each.

Keep the receipts

When an agent makes a decision or calls a tool, that should be logged in an audit trail you can reconstruct later. Audit trails turn governance from a blocker into an enabler — they’re what lets you investigate an incident, provide evidence of compliance to a regulator, and decide whether to widen its scope over time.

Name an owner

Every agent needs a human accountable for it. Diffuse ownership is how an agent drifts out of policy with no one noticing. NIST’s framework treats this as part of governance too, calling for documented roles and responsibilities so the right people are responsible for mapping, measuring and managing AI risks. A named owner closes the loop between the inventory, the controls, and the audit trail.

That owner does not define every permission alone. Data owners, security and platform teams, application owners, and users each control a different part of the boundary. See agent identity and permissions for the full ownership model.

Practice the owner’s decision

In our fictional agent patch review, an agent can edit the proposed code and tests, while a human controls an independent deployment gate. Inspect two pieces of evidence, then authorize a limited release or keep the patch unmerged. Use the exercise to separate the agent’s explanation from the evidence an accountable owner needs.