Your validation team faces growing demands without an increase in staff. You're tasked with validating traditional credit risk models, business applications, non-model analytical tools, and now generative AI systems. Regulators expect deeper analysis, and executives want cost efficiency. Traditional automation helps with repetitive tasks, but model validation requires more nuanced judgment.
Agentic AI offers a new approach. Unlike rules-based automation, AI agents handle unstructured information, draft Validation Evidence, and propose test plans based on model taxonomy. Banks using traditional automation see 20-30% faster validation times. Adding AI agents can automate workflows end-to-end, with human oversight at key decision points.
This isn't just theory. Santander UK has pipelines that pull from model taxonomies to generate initial validation reports, allowing validators to focus on business context and assumptions instead of data gathering. This shift requires rethinking your validation workflow.
What You Need Before Starting
Foundational automation infrastructure: Before deploying AI agents, ensure your validation function has working pipelines for model inventories, risk taxonomies, and test component libraries. If you're manually gathering model documentation or copying test results into Word, address these basics first.
Standardized taxonomies: AI agents need consistent metadata to route work correctly. Map your models to a taxonomy capturing risk tier, model type (credit, operational, market), deployment status, and regulatory scope. ISO/IEC 5338 provides a useful starting framework.
Code and documentation repositories: Agents will retrieve testing components and reference materials. Ensure you have version-controlled test scripts, validation templates organized by model type, and searchable access to prior validation reports. Consolidate if your validation evidence is scattered across email threads and shared drives.
Pre-deployment testing capability: Before deploying AI tools in your workflow, set up a controlled environment to test them. Access non-production model data, run parallel validations (human vs. agent-assisted), and measure accuracy and consistency.
Validator buy-in: Your team needs to understand they're supervising intelligent systems, not being replaced. With increasing MRM scope and regulatory expectations, you're redirecting effort from execution to oversight and expert judgment.
Step-by-Step Implementation
Phase 1: Configuration and Planning
Deploy AI agents to analyze validation requests and generate initial work plans. The agent reads model metadata from your taxonomy, identifies the model type and risk tier, and proposes a validation approach.
Implementation:
- Configure your agent with access to your model inventory API and validation playbook repository.
- Define prompts that map model characteristics to validation requirements.
- Build a review interface where validators approve, modify, or reject the proposed plan.
Validation checkpoint: Human validators review and approve the plan. The agent doesn't execute until you confirm the scope, test selection, and resource allocation.
Phase 2: Pipeline and Information Retrieval
Once the plan is approved, agents retrieve testing components and pull contextual information from business reports, regulatory filings, and prior validation findings.
Implementation:
- Grant agents read access to your test script repository, model documentation storage, and business intelligence systems.
- Configure retrieval logic for different model types.
- Set up enrichment workflows where agents flag gaps.
Santander UK's approach creates pipelines to model taxonomies that auto-populate first drafts with data to examine and tests to run. Validators decide what's relevant; the agent assembles the raw material.
Phase 3: Execution and Draft Generation
Agents execute standard tests using your automated testing infrastructure, then generate first-draft Validation Evidence.
Implementation:
- Integrate agents with your existing test automation platform.
- Configure document generation: agents insert test results, generate summary statistics, create comparison tables, and draft findings sections.
- Build templates that separate agent-generated content from validator-authored content.
Critical control: Agents execute tests but don't interpret materiality or make approval decisions. If a backtesting exception occurs, the agent flags it. The validator determines whether it's acceptable or requires recalibration.
Phase 4: Quality Assurance and Compliance Checks
Before the draft reaches the validator, agents perform automated quality checks against your validation standards.
Implementation:
- Define quality rules for different model types.
- Configure compliance checks against internal standards and regulatory requirements.
- Set up flagging logic for missing sections.
This increases consistency across validators and reduces the risk of incomplete validations reaching model owners or audit committees.
Phase 5: Validator Review and Approval
The validator reviews the agent-generated draft, adds business context and expert judgment, challenges assumptions, and finalizes the validation report.
Implementation:
- Build a review interface that clearly distinguishes agent-generated content from validator additions.
- Track validator edits to identify where agents consistently need correction.
- Require validators to sign off on the final report, maintaining accountability.
You're not removing the validator from the workflow. You're removing the parts where they're transcribing test results and formatting tables, so they can focus on challenging model assumptions and assessing business context.
Validation: How to Verify It Works
Run parallel validations: For your first 10-15 models, run both traditional and agent-assisted validations in parallel. Compare turnaround time, consistency of test coverage, and quality of findings.
Measure agent accuracy: Track how often validators accept agent-generated plans without modification and how frequently agent-drafted content makes it into the final report unchanged.
Monitor for tool dependency: Ensure you're not using the same AI tools for model development and validation. You need an effective second line of defense, not an echo chamber.
Test agent failure modes: Deliberately feed your agents edge cases to see where they break down and build guardrails.
Maintenance and Ongoing Tasks
Continuous monitoring: Set up dashboards tracking agent performance: accuracy rates, validator override frequency, cycle time trends, and error patterns.
Regular upskilling: Keep your validators fluent in the AI tools they're supervising. Schedule quarterly training on new agent capabilities and emerging AI risks.
Agent retraining: Use validator feedback to retrain your agents quarterly.
Taxonomy updates: Update your taxonomies and agent prompts when you add new model types or regulatory requirements change.
Tool diversification: Review your AI tool stack annually to ensure you're not over-reliant on a single vendor.
The goal isn't full automation, it's intelligent augmentation. Validators supervise multi-agent systems instead of executing every workflow step manually. They focus on judgment, context, and challenge rather than data gathering and document formatting. Given the expanding scope of MRM and rising regulatory expectations, this is a capability expansion strategy.
