Your model risk framework was built for predictable systems. Generative AI doesn't work that way. If you're treating LLMs and diffusion models like traditional statistical models, you're missing risks that matter.
This checklist translates SR 11-7 principles and emerging AI governance standards into concrete steps for generative AI. It's not exhaustive, but it covers the control gaps most teams discover too late.
Prerequisites
Before you start, confirm you have:
Authority to classify systems. Someone must decide whether a generative AI application qualifies as a model under SR 11-7 or as a high-risk AI system under the EU AI Act. Don't assume every chatbot interface needs full model validation.
Access to technical artifacts. You need model cards, training data documentation, fine-tuning records, and API specifications. If you're using a Foundation Model Provider's API, get their technical documentation and terms of service now.
A risk tiering method. Your existing framework probably uses quantitative thresholds (dollar exposure, customer count). Generative AI introduces qualitative risks: reputational harm from hallucinations, copyright exposure from training data, bias in synthetic content. Adapt your AI RMF Profile to handle both.
Checklist Items
1. Document Model Scope and Limitations
What to check: The model card or system card clearly states what the model can and cannot do, using plain language a business owner would understand.
Requirement: ISO/IEC 42001 Annex A Control 6.2.3; NIST AI RMF Govern function.
Good looks like: "This summarization model works on English financial documents up to 10,000 tokens. It fails on tables, mathematical proofs, and non-English text. Do not use for regulatory filings without human review."
2. Identify and Classify Prohibited Use Cases
What to check: You've reviewed the EU AI Act's prohibited AI practices and confirmed your application doesn't fall into banned categories (social scoring, emotion recognition in workplaces, biometric categorization).
Requirement: EU AI Act Article 5.
Good looks like: A signed attestation from the business owner and legal counsel stating the system's purpose and confirming it doesn't perform prohibited functions.
3. Validate Training Data Provenance and Licensing
What to check: You know where the training data came from. If you fine-tuned a foundation model, you've documented your training set. If you're using a third-party model, you've asked the Foundation Model Provider about their data sources and licensing.
Requirement: EU AI Act Technical Documentation (Annex IV); GDPR Article 5.
Good looks like: A data lineage document showing dataset origin, licensing terms, and confirmation that no copyrighted material was used without authorization. For vendor models, a contractual representation covering training data compliance.
4. Test for Hallucination Rates in Your Domain
What to check: You've run domain-specific test cases and measured how often the model produces false but confident-sounding outputs.
Requirement: SR 11-7 validation principle; ISO/IEC 23894 risk assessment.
Good looks like: A validation report showing hallucination frequency across 500 representative prompts, with severity ratings and business impact analysis. You've set a tolerance threshold and the model meets it.
5. Implement Disclosure of AI Interaction
What to check: Users know they're interacting with an AI system, not a human.
Requirement: EU AI Act Article 52; General-Purpose AI Code of Practice transparency obligations.
Good looks like: A clear notice at the start of every AI-generated interaction: "This response was generated by an AI system. Verify critical information independently."
6. Establish Reproducibility Controls for Deterministic Outputs
What to check: You can reproduce model outputs when needed for audit or dispute resolution. You've documented temperature settings, random seeds, and API versioning.
Requirement: SR 11-7 validation evidence standards; ISO/IEC 5338 AI lifecycle processes.
Good looks like: A configuration management log showing model version, hyperparameters, and prompt templates for every production deployment. You've tested output consistency and documented acceptable variance ranges.
7. Define Human Oversight Requirements
What to check: You've specified when and how humans review AI outputs before they affect decisions.
Requirement: EU AI Act Article 14; SR 11-7 model governance.
Good looks like: A decision matrix: "Loan summaries under $10K: spot-check 5%. Over $10K: full review. Customer-facing communications: mandatory review before sending."
8. Assess Vendor Model Risk and Contractual Protections
What to check: If you're using a third-party API, you've conducted vendor due diligence and negotiated appropriate Service Level Agreements, indemnification, and model change notification terms.
Requirement: SR 11-7 vendor model risk principles; ISO/IEC 42001 Annex A Control 5.3.
Good looks like: A vendor risk assessment covering model performance guarantees, data handling, change management, and liability allocation. You've negotiated 30-day notice for model updates that could affect outputs.
9. Implement Content Filtering and Rate Limiting
What to check: You've deployed controls to prevent misuse: input filtering for prompt injection attacks, output filtering for harmful content, and rate limiting to prevent abuse.
Requirement: General-Purpose AI Code of Practice; ISO/IEC 23894 risk treatment.
Good looks like: Technical controls that block known prompt injection patterns, scan outputs for personally identifiable information and offensive content, and limit requests to 100 per user per hour.
10. Establish Post-Market Monitoring for Drift and Degradation
What to check: You're tracking model performance in production, not just at deployment. You've defined metrics, set alert thresholds, and assigned monitoring responsibility.
Requirement: EU AI Act Article 72; SR 11-7 ongoing monitoring.
Good looks like: A monitoring dashboard tracking output quality scores, user feedback, hallucination reports, and task completion rates. Monthly performance reviews with defined escalation procedures when metrics degrade.
Common Mistakes
Treating foundation model APIs as black boxes. You're still accountable for outputs even if you didn't train the model. Demand technical documentation from your Foundation Model Provider.
Skipping adversarial simulation. Traditional validation tests normal use cases. Generative AI needs red teaming: deliberately trying to break the system with adversarial prompts.
Validating once at deployment. These models change: through fine-tuning, through vendor updates, through shifts in user behavior. Your validation evidence expires.
Ignoring copyright and IP exposure. If your model was trained on copyrighted content or generates outputs that infringe, you own that risk. Legal review isn't optional.
Next Steps
Run through this checklist for each generative AI system your organization deploys. Document gaps, assign owners, and set remediation deadlines.
Then update your model risk management framework with generative AI-specific validation standards. Your existing SR 11-7 program gives you the governance structure; you need to adapt the technical validation methods.
Start with your highest-risk applications: customer-facing systems, anything touching regulated decisions, and any tool that generates content under your brand. Work backward from there.
The controls exist. You just need to apply them before your first incident forces the conversation.
