Does Your GenAI System Need Validation or Verification?

Your team has received a request to deploy a large language model for drafting customer communications. Your current validation protocol requires training data lineage, performance metrics on a holdout set, and stability testing across known input ranges. The requestor mentions the model was fine-tuned on a licensed foundation model, the training data is proprietary, and performance is judged subjectively by business users.

You're facing a key governance question: does this system need the same validation rigor as your traditional credit scoring models, or does it require a different approach?

The Decision You're Facing

Traditional model validation, as outlined in SR 11-7, assumes you can test a model's outputs against known ground truth, measure error rates, and verify reproducibility. Generative AI systems often lack these anchors. You're deciding whether to:

• Modify your existing validation framework
• Develop a parallel verification process for GenAI systems
• Create a hybrid approach that gates deployment based on system characteristics

This isn't just a semantic difference. Validation implies mathematical proof of accuracy within tolerance bands. Verification confirms the system behaves as intended for its use case, even when "correct" outputs can't be precisely defined.

Key Factors That Affect Your Choice

1. Output determinism
If your GenAI system produces the same output given identical inputs (temperature set to zero, no sampling), traditional validation methods apply more cleanly. If outputs vary by design, you need verification methods that assess output distributions rather than point predictions.

2. Existence of ground truth
Can you define "correct" answers for your use case? A GenAI system summarizing medical records for prior authorization has verifiable facts. A system generating marketing copy has style preferences, not correctness.

3. Regulatory classification
The EU AI Act designates certain AI systems as high-risk based on use case (employment decisions, credit scoring, law enforcement). High-risk systems trigger Technical Documentation (Annex IV) requirements and conformity assessment. If your GenAI system falls into Annex III categories, validation rigor increases regardless of technical architecture.

4. Model provenance
Did you train the model end-to-end, fine-tune a foundation model, or deploy a third-party system via API? Your validation scope changes dramatically. Foundation Model Providers may not share training data or architecture details, limiting your ability to perform traditional validation.

5. Human-in-the-loop design
Systems designed with human review before consequential action require less validation rigor than fully automated decision systems. If your GenAI drafts recommendations that a loan officer reviews and edits, you're verifying decision support, not validating autonomous decisioning.

Path A: Traditional Validation with GenAI Adaptations

Choose this path when:

• Your GenAI system makes decisions with quantifiable outcomes (approve/deny, classify, predict a number)
• You have access to training data and model architecture
• You can construct test datasets with known correct outputs
• The system operates within a defined input domain
• SR 11-7 or equivalent model risk management standards apply to your organization

What this looks like:
Adapt your existing validation protocol to address GenAI-specific challenges. For a fine-tuned model that classifies support tickets, you'd:

• Document the base model's training methodology and any known limitations
• Test the fine-tuned model's performance on a holdout dataset with labeled ground truth
• Measure performance degradation on out-of-distribution inputs
• Conduct adversarial simulation to probe for vulnerabilities
• Establish monitoring thresholds for output quality metrics (classification accuracy, F1 scores)
• Require reproducibility documentation: can you recreate the fine-tuning process and achieve comparable results?

You're still producing validation evidence that ties model performance to risk tolerances, but you're expanding your test battery to cover GenAI attack surfaces.

Path B: Verification-Focused Governance

Choose this path when:

• Outputs are subjective or creative (content generation, summarization, translation)
• No single "correct" answer exists for most inputs
• You're consuming a foundation model via API with limited visibility into training
• The system augments human judgment rather than replacing it
• Use case doesn't trigger high-risk AI classification under the EU AI Act

What this looks like:
Verify the system meets functional requirements and operates within acceptable risk bounds, without claiming mathematical validation. For a GenAI system that drafts internal policy summaries:

• Define instructions for use: what tasks is the system designed for, and what's out of scope?
• Establish output quality criteria through stakeholder engagement
• Conduct structured human evaluation: do subject matter experts judge outputs as fit for purpose?
• Test for prohibited AI practices: does the system produce outputs that violate policy or regulation?
• Implement disclosure of AI interaction so users know they're reading AI-generated content
• Set up post-market monitoring for output quality drift and user feedback loops
• Document model limitations and use restrictions explicitly

You're not claiming the system is "accurate" in the SR 11-7 sense. You're verifying it performs its intended function within defined guardrails.

Path C: Risk-Tiered Hybrid Approach

Choose this path when:

• You're deploying multiple GenAI systems across varied use cases
• Some systems make consequential decisions; others provide decision support
• You need a scalable governance framework that doesn't bottleneck innovation
• You want to align with NIST AI RMF risk tiering principles

What this looks like:
Classify GenAI systems into tiers based on impact and apply proportional rigor:

Tier 1 (High impact, regulatory scrutiny): Full validation per Path A. Examples: credit underwriting assistance, fraud detection, HR screening tools. Require validation evidence, third-party validation for outsourced models, continuous performance monitoring.

Tier 2 (Moderate impact, business risk): Verification per Path B plus targeted validation of critical functions. Examples: contract analysis tools, research summarization, code generation for internal use. Require AI system impact assessment, human review protocols, and Root Cause Analysis analysis.

Tier 3 (Low impact, experimental): Lightweight verification focused on safety and policy compliance. Examples: brainstorming assistants, internal knowledge search, draft generators with mandatory human editing. Require instructions for use, basic output review, and user training.

Your AI management system (per ISO/IEC 42001) defines the tiering criteria and assigns accountability for classification decisions. Document the rationale in Technical Documentation (Annex IV) for high-risk systems.

Summary Matrix

Factor	Traditional Validation (Path A)	Verification (Path B)	Hybrid (Path C)
Best for	Quantifiable decisions, regulated use cases	Subjective outputs, decision support	Portfolio of varied GenAI systems
Key artifact	Validation evidence with performance metrics	Verification report with functional acceptance	Risk tier assignment + proportional documentation
Ground truth requirement	Must have labeled test data	Quality criteria via expert judgment	Varies by tier
Regulatory alignment	SR 11-7, high-risk EU AI Act systems	ISO/IEC 42001 controls, moderate-risk systems	NIST AI RMF, scalable governance
Effort level	High: full model testing and documentation	Moderate: functional testing and monitoring	Variable: allocate rigor to risk
Model access needed	Architecture, training data, retraining capability	API access, system behavior documentation	Depends on tier and use case

The choice isn't binary. Your credit risk models still need SR 11-7 validation. Your customer service chatbot needs verification that it doesn't produce harmful outputs and escalates appropriately. The framework you choose should match the risk you're managing, not the technology label on the system.