AI Governance That Actually Works: If You Can't Enforce It, It Isn't Governance

AI Governance That Actually Works: If You Can't Enforce It, It Isn't Governance

Table of Contents

Here’s a question I’d put to any board that has signed off an AI policy: “If an AI agent broke that policy this afternoon, how would we know, and what would stop it?”

For most organizations, the honest answer is that they wouldn’t, and nothing would.

That’s not because the policy is badly written. Many AI governance policies are thoughtful, well-aligned with frameworks and approved at the highest level. The problem is that they live in documents, while AI agents live in production. A policy that says “only approved AI tools may access customer data” means very little if nothing checks which tools are approved, and nothing stops the ones that aren’t.

This is the final article in a series on securing agentic AI. The earlier pieces covered the MCP protocol, MCP gateways, the internal AI marketplace and agent-to-agent security. This one pulls them together around the question boards actually care about: are we in control?

The AI governance problem in plain English

The major frameworks agree on what good looks like. The NIST AI Risk Management Framework organizes it into governing, mapping, measuring and managing AI risk. ISO/IEC 42001 turns it into a certifiable management system. The EU AI Act adds legal obligations such as record-keeping and human oversight for higher-risk uses.

What the frameworks can’t do is make those things happen in your systems. That’s the job of architecture.

So test your own governance against five practical questions:

  1. Do we have a complete, current list of the AI agents, tools and skills in use, each with an owner?
  2. Can we show who approved each one, and what checks it passed?
  3. Is access limited to the people and agents who need it, and is that enforced, not just documented?
  4. Can we prove what a given AI did, on whose behalf, last month?
  5. Can we stop a specific AI component, everywhere, in minutes?

If the evidence for these lives in spreadsheets and email threads, your governance is a description of intent. It’s not yet a control.

Let me paint a picture

Think about how companies govern employee spending.

Every company has an expense policy. It says sensible things: no personal purchases, stay within limits, get approval for anything large. Years ago, that policy was enforced after the fact. People spent money, submitted receipts and hoped someone checked.

Then came corporate cards with controls built in. The card declines at merchant categories you’ve blocked. It has a spending limit per person. Large purchases need approval before they go through. Every transaction is recorded against a named employee, automatically. And if a card is lost, it’s frozen in seconds.

The policy didn’t change. What changed is that the policy started to enforce itself, and to produce its own evidence.

Most AI governance today is still at the “submit your receipts” stage. Agents are spending authority on your behalf, and we’re hoping someone checks.

Now what?

Why policies and committees aren’t enough

AI governance boards, acceptable use policies and risk assessments are all necessary. On their own, they fall short for two reasons.

Agents act faster than committees meet. An agent can call hundreds of tools an hour. Governance that relies on periodic review will always be looking at yesterday’s risk, while today’s actions go unchecked.

Paper controls produce no evidence. When an auditor or regulator asks you to demonstrate that only approved AI accessed personal data, a policy document isn’t proof. Logs, approval records and access decisions are.

The fix isn’t more policy. It’s connecting the policy you have to controls that enforce it automatically and record that they did.

AI governance principles that hold up in production

If you only take one section of this article back to your team, take this one.

  1. If you can’t enforce it, it isn’t governance. Every policy statement should map to a technical control, or be flagged as a gap.
  2. One system of record for AI. Every agent, tool and skill is registered in one place, with an owner, a purpose and a status.
  3. Approval is a gate, not a form. Nothing becomes usable until it has passed your checks, and the approval is recorded automatically.
  4. Least privilege by default. People and agents get access to specific tools and agents, not everything.
  5. Every action is attributable. Each AI action traces back to a named person or a named agent, never a shared account.
  6. Change is governed too. Updates, new versions and content changes trigger checks, not silence.
  7. Evidence is a by-product. Controls should produce audit-ready records as they run, so compliance isn’t a separate project.

From policy to control: the governance architecture

The good news is that if you’re building the platform described in this series (a registry, a gateway and agents with their own identities), you already have most of the enforcement points you need. Governance is about connecting them to policy.

flowchart TB
    POL["AI policy and risk appetite
(board-approved)"] --> CTRL["Enforceable controls"] CTRL --> REG["Registry
inventory, approval, trust, visibility"] CTRL --> GW["Gateway
identity, least privilege, audit"] CTRL --> AG["Agents
initiator checks, human approvals"] REG --> EV["Evidence
approvals, scans, access decisions, logs"] GW --> EV AG --> EV EV --> REV["Oversight: metrics, audits,
incident review"] REV -->|"adjust"| POL

Here’s how each governance question maps to a control. The examples draw on the gateway and registry my team built for our AI platform, but the pattern applies to any platform.

Governance questionControl that answers itEvidence it produces
What AI do we have?A registry of every agent, MCP server and skill, with owners, versions and lifecycle status, kept current by auto-discoveryA live inventory, not a spreadsheet
Who approved it?An admission gate wired to your approval workflow, plus automated security scanningApproval decisions and scan reports per item
How much do we trust it?Trust levels and verified identity for agentsA clear label on every item
Who can use it?Group-based, per-tool permissions and per-item visibilityAccess rules you can review and export
What did it do?Gateway audit logs tied to named people and agentsA searchable record of every tool call
Has it changed?Versioning, content fingerprints and scheduled rescansChange and drift alerts
Can we stop it?Central disable, token revocation and session invalidationA timestamped record of the response

1. Inventory: you can’t govern what you can’t see

The registry becomes your AI system of record. Every MCP server, agent and skill is registered with an owner, a description, a version and a lifecycle status (draft, beta, active or deprecated). Unregistered components don’t work, because the gateway only routes to what’s registered. In our platform, MCP servers running in our clusters are also discovered automatically, so the inventory reflects what’s actually deployed rather than what someone remembered to record.

That turns the first question every framework asks, “what AI systems do you have?”, from a survey into a query.

2. Approval: governance at the point of entry

An admission gate should send every new registration, including anything found by auto-discovery, to an approval service you control, which returns allow or deny and fails closed if it can’t be reached. Combined with automated security scanning, where anything with serious findings is held for review, you get a consistent entry process that records its own decisions.

This is where your risk tiers belong. Low-risk items, like a read-only documentation search, can be approved automatically. Anything touching personal data, money or production systems goes to a human reviewer.

3. Access: least privilege you can prove

Permissions map identity provider groups to specific servers, tools and agents. Access reviews become a review of those mappings rather than a hunt through dozens of systems. And because the gateway enforces them on every call, the access you document is the access that exists.

4. Accountability: every action has a name on it

This is where many AI deployments quietly fail governance. If ten developers share one service token, the audit log says “the service did it”, and accountability disappears.

Modern AI assistants can sign each user in individually. The MCP specification defines standard OAuth discovery for exactly this, and a gateway that publishes those documents lets compliant assistants sign the user in through your identity provider, so every call carries that person’s identity and group permissions. Where a shared credential is unavoidable, issue short-lived tokens per person instead, narrowed to what the task needs. Autonomous agents get their own identities. Either way, the audit trail names a person or a specific agent.

Two practical notes:

  • Check your retention. Audit logs are often kept for days by default. That’s useful for debugging and useless for a regulator asking about last quarter. Export logs to your SIEM and retain them in line with your obligations.
  • Keep humans in the loop where it counts. For high-impact actions, require an explicit human approval and record it as verifiable evidence, as described in Zero Trust for AI Agents.

5. Change and response: governance that keeps up

Approvals go stale when things change. Versioned servers (our gateway can run several versions of a server side by side, with instant rollback), content fingerprints for skills and scheduled rescans turn “has anything changed since we approved it?” into an alert rather than a guess.

And when something goes wrong, you need to act at the speed of the problem: disable a component everywhere (every item in our registry has a single enable switch for exactly this), revoke its access and, if needed, invalidate every active session. Those runbooks should exist, be tested and produce a record, just like any other incident response.

How this answers the board

Let’s go back to the corporate card.

The expense policy is still there. But now it enforces itself: blocked categories, per-person limits, approvals before large purchases, every transaction named and a lost card frozen in seconds. And at month end, the evidence is simply there.

For AI, that gives you:

  1. A live inventory of every AI component, with owners.
  2. Recorded approvals that combine your policy with security checks.
  3. Enforced least privilege, not documented intent.
  4. Named accountability for every AI action.
  5. The ability to stop anything, fast, with evidence of what you did.

And it gives the board metrics that mean something: the percentage of AI components registered and approved, items currently held for security review, the percentage of AI actions attributable to a named person, and how long it takes to shut down a component when you need to.

Our AI policy is only as strong as the controls that enforce it, and the evidence that proves they worked.

What this doesn’t do

  • It doesn’t decide your risk appetite. Controls enforce policy; they don’t write it. Leadership still has to decide what’s acceptable.
  • It doesn’t cover AI outside the platform. Employees using consumer AI tools, AI features built into SaaS products and models embedded in vendor systems all need their own governance routes.
  • It doesn’t assess model quality or fairness. Bias, accuracy and explainability need separate evaluation and testing processes. This architecture governs access, actions and accountability.
  • It doesn’t make you compliant by itself. Frameworks and regulations require processes, roles and documentation too. What this gives you is the evidence those processes depend on.

How I’d roll out an AI governance framework

  1. Phase 0: Map policy to controls. Take each statement in your AI policy and identify the technical control that enforces it. Mark the gaps honestly.
  2. Phase 1: Build the inventory. Stand up the registry, bring in existing components and assign an owner to each.
  3. Phase 2: Wire approvals to risk tiers. Connect the admission gate to your review process, with automatic approval for low-risk items and human review for high-risk ones.
  4. Phase 3: Make identity personal. Replace shared tokens with per-user sign-in and per-agent identities, and route tool access through the gateway.
  5. Phase 4: Produce evidence. Export audit logs, approval records and scan results to your SIEM and GRC tools, with retention to match your obligations.
  6. Phase 5: Report and rehearse. Give the board a small set of meaningful metrics, and run a tabletop exercise on shutting down a misbehaving agent.

The trade-offs to watch

  • Bureaucracy creep. If every change needs a committee, teams will route around the platform. Automate low-risk approvals and invest review time where the risk is.
  • Evidence volume. Logging every tool call produces a lot of data. Decide what you need to retain, for how long, and what must be kept away from personal data.
  • Ownership over time. Governance depends on named owners, and people change roles. Build ownership transfer into your joiner, mover and leaver process.
  • Framework chasing. Frameworks and regulations will keep evolving. Anchor on controls and evidence, then map them to each framework, rather than rebuilding for every new one.

What to ask your team on Monday

  1. For each statement in our AI policy, what technical control enforces it?
  2. Can we produce a current list of every AI agent, tool and skill in use, with an owner for each?
  3. What percentage of AI actions last month can we attribute to a named person or a specific agent?
  4. How long do we keep AI audit records, and does that match our regulatory obligations?
  5. If we needed to shut down a specific AI component everywhere today, how long would it take, and have we ever tried?

AI governance has spent its first chapter in documents, which was the right place to start. The next chapter is in the architecture. The organizations that get this right won’t have longer policies. They’ll have policies that enforce themselves, and evidence ready when someone asks.

If you can’t enforce it, it isn’t governance. Make every policy statement a control.

Frequently asked questions

What is an AI governance framework?

An AI governance framework is the set of policies, roles, processes and controls an organization uses to manage AI risk. Effective frameworks connect each policy statement to a technical control that enforces it and produces evidence.

How do NIST AI RMF, ISO/IEC 42001 and the EU AI Act relate?

The NIST AI Risk Management Framework is voluntary guidance for managing AI risk. ISO/IEC 42001 is a certifiable AI management system standard. The EU AI Act is law, with obligations based on risk level. Many organizations use NIST and ISO to help meet the Act’s requirements.

How do you govern AI agents in production?

Keep an inventory of every agent and tool with an owner, gate what gets approved, give each agent its own identity with least-privilege access, log every action against a named person or agent, and be able to disable any component instantly.


If you’re turning an AI policy into enforceable controls and want to pressure-test your approach, book a strategy session.

Share :