The Conventional Wisdom
There's a growing belief that generative AI (GenAI) will revolutionize model risk management by automating validation, catching errors human reviewers might miss, and solving resource constraints. Vendors often pitch AI-assisted documentation review, automated testing protocols, and intelligent monitoring systems that continuously validate model performance. The logic seems sound: validation is time-consuming, requires specialized expertise, and struggles to keep up with the pace of model deployment. So why not apply AI to the problem?
Why We Disagree
This perspective misses the real challenge. The issue isn't whether GenAI can perform validation tasks; it's that using GenAI for validation introduces a new layer of model risk that you're not prepared to handle. Deploying a GenAI system for validation means you now have a model that itself requires validation under SR 11-7. This validation must be done by qualified personnel who are independent from the model's development. You can't validate the validator with itself.
The resource constraint doesn't disappear; it shifts. Instead of needing validation expertise just for your business models, you now need it for both your business models and your validation tools. This adds complexity rather than reducing it.
Consider the validation requirements under SR 11-7: conceptual soundness analysis, ongoing monitoring, and outcomes analysis. For a GenAI validation tool, this means understanding the foundation model's training data, architecture, and known failure modes; documenting prompt engineering and testing stability across model updates; establishing performance benchmarks for validation tasks; and monitoring for drift in the GenAI system's outputs, not just the models it's reviewing.
Most organizations struggle to validate deterministic credit models. Adding probabilistic, black-box GenAI systems to your validation workflow complicates things further.
The Evidence
The EU AI Act classifies AI systems used in "safety components" as high-risk. A GenAI system performing validation in a regulated environment falls into this category, subjecting you to conformity assessment requirements, technical documentation obligations under Annex IV, and post-market monitoring mandates.
ISO/IEC 42001's Annex A controls require you to establish competence requirements for AI system operation (Control 6.2.2) and maintain traceability throughout the AI lifecycle (Control 6.3.5). When your validation process depends on GenAI, you need documented competence in model risk management, prompt engineering, foundation model behavior, and AI system monitoring. Your validation evidence must now trace through two systems instead of one.
The NIST AI RMF's Measure function asks you to assess AI risks and performance. Using GenAI to measure other models means you're using an instrument that itself requires measurement. The framework doesn't exempt tools from governance requirements just because they're used for governance purposes.
The practical problem is validation independence. You can't have the same team develop a model and validate it. When you use GenAI for validation, who validates the GenAI system? If it's the same model risk team, you haven't achieved independence. If it's a separate team, you've doubled your validation headcount requirement.
What to Do Instead
Focus GenAI applications on documentation assistance and structured data extraction, not validation judgment. These use cases add value without introducing circular validation dependencies.
Use GenAI to draft model documentation templates, extract information from vendor technical specifications, or summarize changes between model versions. These tasks support your validation team but don't replace validation judgment. A human reviewer still makes the conceptual soundness determination. The GenAI system just speeds up information gathering.
Establish clear boundaries. Your validation framework should specify which tasks require human expert judgment (conceptual soundness review, materiality assessment, approval decisions) and which support activities can use AI assistance (documentation formatting, reference lookup, change tracking). Document this in your model risk management policy.
If you deploy GenAI for validation support, treat it as a high-risk AI system from day one. Build the validation framework before you deploy the tool. Establish performance benchmarks, define monitoring metrics, and document model limitations and use restrictions. Don't retrofit governance after embedding the system in your validation workflow.
Invest in your validation team's AI literacy instead of AI validation tools. Your validators need to understand foundation model behavior, prompt sensitivity, and AI system failure modes because they'll be validating AI models across your organization. That expertise delivers more value than any automated validation assistant.
When the Conventional Wisdom Is Right
GenAI has a role in model risk management, just not in replacing validation judgment. The conventional wisdom correctly identifies that validation teams face resource constraints and that AI can process information faster than humans. Where it errs is in assuming speed equals better validation.
Use GenAI when you need to process large volumes of structured information quickly. For example, a team managing hundreds of models that needs to track which models reference deprecated data sources could use GenAI to scan documentation and flag potential issues faster than manual review. The validation team still investigates each flagged item and makes the risk determination.
The conventional wisdom also correctly notes that GenAI can identify patterns humans might miss. This works when analyzing model monitoring data for anomalies or comparing model behavior across similar use cases. The AI highlights patterns; your validators interpret Materiality and determine appropriate action.
If you're a foundation model provider subject to the General-Purpose AI Code of Practice, you'll likely need AI systems to help manage systemic risk obligations at scale. That's a different calculation. You're already operating in a high-AI-risk environment with mature AI governance capabilities. For most organizations managing model risk under SR 11-7, that's not your reality.
The promise of AI-automated validation isn't wrong because the technology can't do it. It's wrong because doing it well requires more governance infrastructure than most teams can support. Start with use cases that assist validation without requiring validation themselves.
