Identifying the Problem
In governance working sessions over the past six months, a recurring issue has emerged. Teams map AI systems to risk tiers, document controls in their ISO/IEC 42001 AIMS, and conduct quarterly reviews. Yet, models still drift, fail unexpectedly, or are used in unforeseen ways. Compliance boxes are checked, but the desired outcomes aren't achieved.
The questions below arise from these sessions. They're what governance leads ask when they realize their risk management framework identifies problems but doesn't prevent them. If you're running an AI governance program, you've likely asked some version of these yourself.
Q1: What's the Difference Between Managing Risk and Controlling Capability?
Risk management identifies potential issues and assigns ownership. Capability control restricts what the system can do from the start.
Traditional model risk frameworks like SR 11-7 and the NIST AI RMF focus on assessment: classifying the model, documenting its limitations, setting monitoring thresholds, and escalating when something drifts. This approach is reactive, measuring risk after deployment and hoping controls catch issues before they matter.
Capability control, on the other hand, defines the boundaries of what the model can do. It specifies which features it can access, what decisions it can make autonomously, and which outputs require human review. These boundaries are enforced at the infrastructure level. For example, rate limiting isn't just about cost; it's about preventing a model from processing more requests than your validation evidence supports. Feature store access controls aren't just security measures; they're capability constraints that keep a model from using data it wasn't validated against.
While risk frameworks might tell you the model could hallucinate in edge cases, capability control prevents deployment in contexts where hallucinations matter.
Q2: Our NIST AI RMF Profile Includes Controls. Isn't That Capability Control?
No. Your AI RMF Profile defines risk tiering and assigns controls to functions (Govern, Map, Measure, Manage). These controls are checkpoints, governance reviews, documentation requirements, testing protocols. They don't enforce operational boundaries.
Consider this gap: your Profile might require "ongoing monitoring of model outputs for bias" (Measure function). That's a detective control. Capability control would restrict the model from making final decisions on protected-class attributes without human override. One observes; the other constrains.
Examine your current controls and ask: does this prevent the behavior, or does it detect the behavior after it happens? If it's all detection, you're managing risk, not controlling capability.
Q3: How Do We Implement Capability Controls Without Breaking Our Deployment Pipeline?
Start with Model Provisioning. When you deploy a model, you're not just pushing code, you're granting access to data, compute, decision pathways, and user interactions. Each of these is a capability you can bound.
Three practical entry points:
Instructions for Use as enforceable constraints. Your Technical Documentation (Annex IV) under the EU AI Act or your model card already describes intended use, limitations, and contexts where the model shouldn't operate. Translate those into deployment rules. If your validation evidence covers customer service queries but not account closure decisions, configure your routing logic to block the model from that pathway.
Feature Store permissions. If a model is validated using demographic data for fraud detection, it shouldn't have runtime access to demographic features for credit decisioning. Your feature store already has access controls, use them to enforce scope.
Rate Limiting tied to validation scope. Your model was validated at 10,000 inferences per day under specific data distributions. If usage spikes to 50,000, you're operating outside your validation evidence. Set rate limits that align with your testing envelope, not just your infrastructure capacity.
None of this requires re-architecting your ML platform. It requires treating deployment configuration as a governance artifact, not just an ops task.
Q4: Who Owns Capability Control, Risk, Engineering, or Governance?
All three, but governance defines the requirements.
Your AI governance team (or whoever owns your ISO/IEC 42001 AIMS) sets the policy: which capabilities require restriction, what the boundaries are, and how they map to risk tiers. Engineering implements those boundaries in the deployment layer. Model risk validates that the controls work and that they're consistent with the model's limitations.
This is why Annex A Controls in ISO/IEC 42001 include both organizational (governance) and technical (engineering) measures. Capability control lives at that intersection.
Practically: governance writes the capability matrix (what the model can and can't do), engineering builds the enforcement mechanisms, and risk audits both against the validation evidence. If those three groups aren't communicating during Model Provisioning, your controls won't hold.
Q5: Do We Need to Re-Validate Every Model?
Not re-validate, re-scope your validation evidence.
Traditional model validation under SR 11-7 focuses on performance: accuracy, stability, sensitivity analysis. Capability control asks a different question: what operational boundaries did we test, and are we enforcing those boundaries in production?
Review your existing validation reports. They already document assumptions: data sources, input ranges, decision thresholds, use cases tested. Extract those assumptions and compare them to your current deployment configuration. If the model is running in contexts you didn't validate, you have two options: expand your validation to cover those contexts or restrict the deployment to match your evidence.
You don't need to re-test models operating within their validated scope. You need to stop deploying them outside that scope without additional validation.
Q6: How Does This Fit with EU AI Act Compliance?
The EU AI Act requires you to document Model Limitations and Use Restrictions (Article 13 for high-risk systems). That's a disclosure requirement. Capability control turns that disclosure into an operational constraint.
If your Technical Documentation (Annex IV) states the model shouldn't be used for decisions affecting vulnerable populations, capability control enforces that by blocking deployment in those contexts. If your General-Purpose AI Code of Practice commits to preventing misuse, capability control is how you operationalize that commitment.
Post-Market Monitoring under the AI Act also shifts from passive observation to active control. You're not just logging incidents, you're using those incidents to refine capability boundaries and tighten restrictions in future releases.
The Act doesn't explicitly mandate capability control, but it's the most defensible way to demonstrate you're meeting your obligations under Articles 9 (risk management) and 13 (transparency and information to users).
Next Steps
Start with your existing model inventory. For each production model, document:
- What it's validated to do (from validation reports or model cards)
- What it's currently doing (from deployment configs and usage logs)
- The gap between those two
That gap is where capability control applies. Then work backward: what deployment-level restrictions would close that gap? Who needs to approve exceptions? How do you audit compliance?
If you're implementing ISO/IEC 42001, map capability controls to Annex A Controls 6.2.6 (AI system use) and 6.2.7 (human oversight). If you're under SR 11-7, treat capability boundaries as part of your ongoing monitoring and escalation framework.
The shift from risk management to capability control isn't about replacing your existing governance, it's about making it enforceable.
