Scope
This guide focuses on implementing an AI control plane for security and governance teams managing agentic and generative AI workloads. It covers discovery, policy enforcement, observability, and accountability throughout the AI lifecycle. It doesn't address predictive model validation procedures (see SR 11-7 guidance separately) or legal liability frameworks still under regulatory development.
Use this when building enforcement infrastructure, instrumenting agents for production, or proving control effectiveness to audit teams.
Key Concepts and Definitions
AI Control Plane: A system providing standardized telemetry, enforceable policy, continuous monitoring, and auditable governance across AI workloads. It acts as the trust layer for agents, bridging applications and business outcomes.
Shadow AI: Workloads deployed faster than the organization can catalog or govern. Discovery must come before control.
Human-in-the-Lead: An operational model where humans orchestrate agent execution and own results, replacing the passive "human-in-the-loop" role.
Three Lines of Defense:
- Business owner (accountable for workload and its controls)
- Risk and compliance (defines controls)
- Audit (verifies controls worked)
Control Loop: A four-part cycle operationalizing governance:
- Define controls from policy, risk, regulation, and operations.
- Implement them as working guardrails.
- Enforce them wherever workloads run.
- Track enforcement to measure business impact.
Requirements Breakdown
Visibility Requirements
V-1: Asset Discovery
Maintain a current inventory of all AI workloads, including ownership, data sources, and external dependencies.
V-2: Exposure Assessment
Map AI assets to use cases, use cases to risks, and risks to business continuity and IT risk categories.
V-3: Third-Party Identification
Document all Foundation Model Providers, agent vendors, and tool integrations with fallback procedures for API outages.
Control Requirements
C-1: Policy Translation
Convert corporate policy, regulatory requirements (EU AI Act, GDPR), and operational limits (token budgets) into measurable controls.
C-2: Metric Assignment
Every control must have at least one actively monitored metric. A control without a metric exists only on paper.
C-3: Pre-Execution Enforcement
Implement guardrails that block prohibited actions before execution, not just flag them in traces afterward.
C-4: Cross-Stack Coverage
Enforce security controls in security tooling, AI-layer controls through tracing and monitoring, and infrastructure controls at the platform layer.
Evaluation Requirements
E-1: Offline Benchmarking
Test third-party agents against published benchmarks before deployment.
E-2: Production Monitoring
Continuously watch for behavioral drift, applying the same rigor used for predictive model decay.
E-3: Step-Level Inspection
Evaluate agent decisions at every step in a run, not just final outputs, since agents break requests into multiple actions.
E-4: Multi-Objective Metrics
Track separate metrics for guardrail effectiveness, cost, latency, and prompt quality. Don't conflate optimization goals.
Accountability Requirements
A-1: Ownership Assignment
The individual who deploys an agent owns what it does. The agent inherits that person's access controls.
A-2: Bidirectional Tracing
Build instrumentation that traces failures from monitoring dashboards back to specific KPIs and forward to financial exposure.
A-3: Decision Documentation
Maintain evidence of model and agent selection decisions, including alternatives considered and rationale.
A-4: Cross-Portfolio Access
Risk, compliance, and audit teams need read access to the entire AI portfolio regardless of departmental boundaries.
Implementation Guidance
Start with Discovery, Not Policy
Your first step isn't writing a governance framework. It's answering: what AI workloads are running right now?
Build a discovery process that catches:
- Agents deployed through official channels
- Generative AI applications built by business units
- Predictive models still running in production
- Third-party tools called by any of the above
Shadow AI is the norm. Your control plane can't enforce what it can't see.
Implement Governance at the Idea Stage
Don't wait for a submitted use case to trigger review. Integrate governance into AI-assisted development environments so builders get feedback on the first prompt.
When someone starts building:
- Check if an approved use case covers the same ground.
- If yes: provide the required guardrails and approve.
- If no and high-risk: flag risk and compliance before code is written.
This removes risk analysis from the builder's responsibility and prevents rework.
Enforce Before the Action Executes
Tracing tells you what already happened. If an agent leaked sensitive information, the trace confirms it after the damage is done.
You need enforcement that:
- Hunts for patterns indicating prohibited behavior.
- Blocks actions before execution.
- Logs the block with full context for review.
Consider a scenario where an agent attempts to call an external API with customer data. The control plane should catch the attempt, verify it against policy, and either block it or require additional approval before the call completes.
Connect Controls to Business Outcomes
The control loop breaks if you can't prove a control worked or measure its business impact.
Your instrumentation should answer:
- Did this guardrail prevent the failure it was designed to catch?
- What was the business cost of the control (latency, token usage, blocked legitimate requests)?
- Does the threshold need adjustment?
Without this feedback, you're running blind. Leadership can't assess whether AI assets deliver on their funding, and audit can't verify effectiveness.
Treat Token Cost as a Control Problem
Operational concerns don't read as compliance risk, but an agent burning through its token budget is still a control failure.
Implement rate limiting and budget enforcement the same way you implement security controls: with a metric, a threshold, and automated enforcement.
Common Pitfalls
Pitfall 1: Governing Only the Final Output
Agents make multiple decisions per run. Evaluating only the final answer misses where things went wrong.
Pitfall 2: Building Controls Without Metrics
A policy document that says "sensitive data must be masked" without telemetry proving it happened is security theater.
Pitfall 3: Treating AI Risk in Isolation
AI risks connect to business continuity and IT risk. If your model provider's API goes down for two minutes, what happens to the business? That's a business continuity question, not an AI-specific one.
Pitfall 4: Stopping at Enforcement
Defining, implementing, and enforcing controls gets you 75% of the way. The missing 25% is tracking whether the control worked and what it cost.
Pitfall 5: Separate Tools for Agentic, Generative, and Predictive AI
Risk and compliance needs one answer covering the entire AI portfolio. Three tools producing three answers creates gaps audit will find.
Pitfall 6: Waiting for External Accountability to Settle
Lawyers, regulators, and business owners don't agree on who's liable when an agent fails. Build internal accountability tracing now. External liability will follow regulation.
Quick Reference Table
| Component | What It Does | Where It Lives | Key Metric |
|---|---|---|---|
| Discovery | Identifies all AI workloads | Asset inventory system | Coverage % of deployed workloads |
| Policy Translation | Converts requirements to controls | Control plane configuration | Controls mapped to requirements |
| Guardrails | Blocks prohibited actions pre-execution | AI layer (tracing/monitoring) | Block rate, false positive rate |
| Step-Level Eval | Inspects agent decisions during runs | Agent orchestration layer | Decisions flagged per run |
| Token Budget | Enforces cost limits | Infrastructure/API gateway | Spend vs. budget, overage frequency |
| Behavioral Monitoring | Detects drift in production | Continuous monitoring platform | Drift detection rate |
| Trace Reconstruction | Maps failures to business impact | Observability platform | Time to root cause |
| Access Control Inheritance | Agents inherit deployer permissions | Identity and access management | Unauthorized action attempts |
| Audit Trail | Proves controls worked | Centralized logging | Completeness %, query response time |
Implementation Priority: Visibility → Control → Accountability. You can't control what you can't see, and you can't prove accountability without both.
Regulatory Note: The EU AI Act's General-Purpose AI Code of Practice and ISO/IEC 42001 both require demonstrable control effectiveness. Build your control plane to produce validation evidence, not just logs.
