When your AI system's capabilities start to surpass your risk controls, you need a documented procedure that your team can execute immediately. This template provides a structured response protocol for when model behavior suggests your safety boundaries aren't holding.
Purpose of the Template
This policy template addresses a specific governance gap: the period between detecting concerning AI system behavior and implementing formal risk controls. It's designed for teams managing General-Purpose AI Models or high-risk AI systems under regulatory frameworks like the EU AI Act or SR 11-7, where documented procedures for capability surprises are necessary.
Use this template when:
- Post-Market Monitoring reveals behavior outside validated use cases.
- Red Teaming exercises uncover capabilities you didn't design for.
- Your model demonstrates reasoning or task completion beyond its documented scope.
- Stakeholder Engagement surfaces use patterns that bypass your intended limitations.
The template creates a decision tree that doesn't require executive approval at every step but ensures you're documenting risk decisions in real time.
Prerequisites
Before customizing this template, ensure you have:
- A current AI System Impact Assessment documenting your model's intended capabilities and risk tier.
- Defined model ownership to authorize deployment changes, data access modifications, or temporary suspension.
- Your organization's risk appetite statement for AI systems, including materiality thresholds.
- Access to your Technical Documentation (Annex IV) if you're EU AI Act-regulated, or your SR 11-7 model documentation package.
- A designated incident response team with at least one person who understands your model's architecture and one who understands your regulatory obligations.
If you lack item #2 or #5, establish those first. This policy can't function without clear authority lines.
The Template
AI SAFETY BREAKOUT RESPONSE POLICY
Version 1.0
1. TRIGGER CONDITIONS
This policy activates when any AI Actor observes:
- Model output that accomplishes tasks outside documented [Instructions for Use](/glossary/instructions-for-use).
- Capability demonstration during Red Teaming that wasn't present in validation.
- User-reported behavior suggesting the model is circumventing [Rate Limiting](/glossary/rate-limiting) or other technical controls.
- [Post-Market Surveillance](/glossary/post-market-surveillance) data showing drift in [Model Limitations and Use Restrictions](/glossary/model-limitations-and-use-restrictions) compliance.
Activation authority: [Role title], with notification to [backup role].
2. IMMEDIATE ACTIONS (within 2 hours)
□ Log the observation in [incident tracking system].
□ Capture: timestamp, user input (if applicable), model output, deployment environment.
□ Notify model owner: [name/role].
□ Assess: Does this behavior create immediate harm risk? If yes, execute suspension protocol (Section 5).
3. ASSESSMENT PHASE (within 24 hours)
Lead: [Model Risk Manager / AI Governance Lead].
□ Compare observed behavior to current AI System Impact Assessment.
□ Determine: Is this a capability expansion, a control failure, or a documentation gap?
□ Review [Reproducibility](/glossary/reproducibility): Can you trigger this behavior consistently?
□ Consult [Foundation Model Provider](/glossary/foundation-model-provider) documentation (if applicable) for known issues.
□ Document [Contextual Risk Factors](/glossary/contextual-risk-factors): What conditions enabled this behavior?
Classification:
[ ] Severity 1: Behavior violates [Prohibited AI Practices](/glossary/prohibited-ai-practices) or creates regulatory breach risk.
[ ] Severity 2: Behavior exceeds risk tier but doesn't violate hard boundaries.
[ ] Severity 3: Behavior is within acceptable risk but wasn't documented.
4. RESPONSE PROTOCOLS
SEVERITY 1:
- Immediate: Suspend model access for affected use cases.
- Within 48 hours: Perform [Root Cause Analysis](/glossary/root-cause-analysis).
- Within 5 business days: Submit incident report to [compliance team/regulator].
- Update Technical Documentation (Annex IV) or SR 11-7 package.
- Re-validation required before restoration.
SEVERITY 2:
- Within 72 hours: Implement compensating controls (enhanced Rate Limiting, modified Instructions for Use, additional [Disclosure of AI Interaction](/glossary/disclosure-of-ai-interaction) requirements).
- Update [Model Cards](/glossary/model-cards) and System Cards.
- Schedule [Model Recalibration](/glossary/model-recalibration) review.
- Notify [Vendor Due Diligence](/glossary/vendor-due-diligence) team if [Outsourced Models](/glossary/outsourced-models) are involved.
SEVERITY 3:
- Update Instructions for Use and user-facing documentation.
- Add behavior to next scheduled validation scope.
- Document in ongoing Post-Market Monitoring log.
5. SUSPENSION PROTOCOL
If immediate suspension is required:
1. [Technical contact] disables [specific API endpoints/model versions].
2. [Communications lead] notifies affected users within [X hours].
3. [Model owner] documents business impact.
4. [Compliance lead] assesses regulatory notification requirements.
5. Restoration requires sign-off from [role] after corrective actions.
6. DOCUMENTATION REQUIREMENTS
Every activation of this policy must produce:
- Incident log entry with observation details.
- Assessment summary with severity classification.
- Response action log with timestamps and responsible parties.
- If Severity 1 or 2: Updated risk documentation reflecting new control state.
7. REVIEW CYCLE
This policy undergoes mandatory review:
- After each Severity 1 activation.
- Quarterly as part of [AI Management System](/glossary/ai-management-system) audit.
- When your organization's risk appetite changes.
- When regulatory requirements change (EU AI Act updates, new guidance).
Policy owner: [Name, title].
Last updated: [Date].
Next review: [Date].
How to Customize It
Section 1 (Trigger Conditions): Add your organization's specific monitoring signals. If you're running continuous Adversarial Simulation, add "simulation results outside expected bounds" as a trigger. If you have automated Bias Mitigation checks, include "Aggregation Bias exceeding threshold X" as a condition.
Section 2 (Immediate Actions): Replace bracketed placeholders with your actual system names and role titles. If you're using a Feature Store with versioning, specify which version metadata to capture. If your models are subject to GDPR, add a trigger to assess whether the incident involves personal data processing outside your Data Protection Impact Assessment scope.
Section 3 (Assessment Phase): Tailor the 24-hour window to your operational reality. Financial services teams under SR 11-7 may need a shorter window for high-risk models. Adjust the classification criteria to match your organization's Risk Tiering methodology from your AI RMF Profile.
Section 4 (Response Protocols): Map these actions to your existing change management and validation processes. If you're ISO/IEC 42001-certified, cross-reference the relevant Annex A Controls you're invoking. If you operate under the General-Purpose AI Code of Practice, add specific transparency obligations for Severity 1 events.
Section 5 (Suspension Protocol): This section needs your actual technical shutdown procedure and communication templates. Don't leave this generic. Your operations team should be able to execute this section without interpretation.
Section 7 (Review Cycle): Align this with your Plan-Do-Check-Act (PDCA) cycle cadence. If you're doing quarterly AI Management System audits, sync the policy review to that schedule.
Validation Steps
Before considering this policy operational:
Run a tabletop exercise. Present your team with a hypothetical scenario: "Your chatbot just explained how to accomplish a task that's explicitly listed in your use restrictions." Walk through each section. Where do people get stuck? Where do they need information that isn't readily available?
Check your authority lines. Can the person designated to activate this policy actually do so? Do they have system access? Budget authority? The ability to convene the assessment team on short notice?
Verify your documentation links. When this policy says "update Technical Documentation (Annex IV)," does your team know where that lives? Who has edit access? What the version control process is?
Test your suspension protocol. In a non-production environment, execute the technical steps in Section 5. Time how long it actually takes. If your policy says "within 2 hours" but the actual process takes 6 hours, adjust your commitments.
Cross-reference your existing policies. Does this policy conflict with your vendor contracts? Your incident response plan? Your change management requirements? Resolve conflicts before you need this policy in production.
This template won't stop your AI systems from developing unexpected capabilities, but it will prevent your team from improvising a response under pressure. Customize it this week, before you need it.
