The enterprise agent conversation stopped being aspirational about twelve months ago. In April 2025 most large deployments were pilots with a human supervisor; by April 2026 we have agents executing complete back-office process steps without human approval for each action. That jump has moved governance from the quarterly committee to the daily operational control, and it has left a set of practices that are no longer optional if you want to pass the next audit or explain an incident to leadership without improvising.
From Committee to Operational Control
Until recently, AI governance at the average enterprise was a policy layer written by a committee and a model register on Confluence. It worked while AI systems were classifiers with a human at the end of the process. With agents that book flights, adjust orders, answer tickets, or modify infrastructure configuration, that documentary layer stopped being enough. The underlying shift is that agents make decisions with real side effects: payments, state changes in systems, emails sent, infrastructure deployed. Each of those effects is a record somebody will want to audit.
The practical consequence is that governance has dropped into engineering territory. A policy stating what cannot be done is no longer sufficient. You need a technical chain that makes forbidden actions impossible, or at least hard. That looks much more like classical identity and access management than like 2023’s ethics committees. Agents are, in practice, non-human identities with credentials, scope, and traceability, and the rigour we apply to service accounts must apply to them.
What a Reasonable Audit Asks Today
A serious internal audit in 2026 walks in with five questions. First: which agents are in production, with what permissions, and who approved them? Answering this demands an agent inventory as strict as the service-account register, with scope, underlying model, callable tools, and validity window.
Second: what has each agent done in the last ninety days? Granular traceability, not just the LLM call log but the full reasoning chain, invoked tools, parameters, result, and side effect produced, has become the most expensive deliverable to stand up and the one auditors ask about first. Typical LLM telemetry is insufficient: you need correlation with target systems.
Third: which guardrails are active, and how do you demonstrate they work? Audits require evidence that controls are not decorative. That means periodic prompt-injection tests, review of cases where the agent tried to step out of scope and how it was blocked, and review of false negatives caught manually.
Fourth: what happens when the agent fails? This is where many projects have no answer. An agent that loops, that burns budget, that produces toxic output, or that executes an irreversible action needs a clear containment path: circuit breaker, human-review queue, alert to the responsible team, reversal procedure.
Fifth: who bears legal responsibility? The EU AI Act in full application since August 2026 requires concrete names on paper for high-risk systems. Model provider, enterprise deployer, compliance owner, and end user have distinct and separable responsibilities, and audits want the map.
What Broke in 2025 and Left a Lesson
The past year produced a catalogue of incidents that explains why governance moved into the operational plane. There are documented cases of agents with corporate-email access executing instructions injected via incoming-message signatures. Support agents, badgered by a persistent user, escalated account permissions without real authorisation. Infrastructure agents, when facing ambiguity, chose the destructive path that solved their task without preserving state.
From those incidents emerged defensive patterns that are now standard. Strict isolation between untrusted content and sensitive tools has prevailed: any agent processing third-party email should not be able to move money or grant permissions in the same context. Temporal scope restriction, where the agent holds broad permissions during its task window and drops to minimum afterwards, has replaced the fixed-account model. And the separation of intent and execution, where one chain proposes and another verifies before acting, has become mandatory for irreversible actions.
Agent Inventory and Minimum Viable Compliance
The minimum a responsible company runs today starts with a central register. Each production agent needs an identifier, function description, base model with version, exposed tools, affected systems, business owner, technical owner, and last security-review date. This register must be alive: if scope changes, it is recorded; if the underlying model changes, it is re-evaluated.
Reasonable policies add an economic guardrail: every agent has a call budget and a monetary budget per unit of time. A looping agent in 2026 costs real money in tokens and in actions on third-party systems, and without a technical limit the risk is unbounded. Most agent-orchestration platforms ship this; the governance job is to define the limits and review them.
agent_id: finance-reconciliation-001
owner_business: finance.ap@company.com
owner_technical: ai-platform@company.com
model: claude-opus-4-7
scope:
tools: [ledger.read, sap.invoice.read, email.draft]
budget_calls_per_day: 2000
budget_usd_per_day: 40
systems: [sap-prod, exchange-corp]
controls:
human_review: actions > 1000 usd
circuit_breaker: errors > 5% in 10 min
audit_log: warehouse.agent_events
last_security_review: 2026-03-12Guardrails Absorbing Most Incidents
After a year in production, three controls account for the bulk of saves. The first is threshold-based mandatory human review: any action above an economic, scope, or reversibility threshold requires explicit human approval regardless of the agent’s declared confidence. The second is an automatic circuit breaker on error rate or anomalous behaviour: if the agent fails too often or drifts from its historical pattern, it suspends itself and alerts. The third is persistent shadow mode: most new agent changes spend weeks running in parallel with the human, without side effects, before being granted real autonomy.
Continuous evaluation has professionalised too. Regression tests on known cases, adversarial prompt-injection tests, and failure drills are part of the standard deployment cycle, as they have been for years with changes to critical applications. The difference is that the input surface is wider and behaviour less deterministic, which forces wider coverage rather than lower frequency.
My Read
Agent governance in 2026 is not a new problem; it is the classic privileged-identity problem with an extra degree of unpredictability. Companies that had fewer incidents are the ones that treated their agents as service accounts with superpowers, not as human users and not as traditional applications. The discipline that worked for IAM still works, with two adaptations: denser traceability, because intermediate reasoning matters, and economic guardrails, because an agent can burn budget without stopping.
If I had to pick three things to start with, they would be a living inventory, threshold-based human review, and an anomaly circuit breaker. Everything else builds on those pillars without surprises. Most of all, resist the temptation to treat governance as friction to reduce: it is the only layer that separates a useful agent from an incident waiting to happen.
