Purpose of the Template
Training a language model on incorrect responses, even in limited contexts, can embed misalignment features that degrade performance across unrelated tasks. This template provides a structured validation script to detect and document these features before they affect your model's behavior.
Use this template when:
- Validating models fine-tuned on domain-specific data
- Assessing Outsourced Models with additional training
- Preparing Validation Evidence for SR 11-7 or ISO/IEC 42001 audits
- Investigating post-deployment performance degradation
The template captures the feature identification process and minimal fine-tuning intervention, structured for compliance documentation.
Prerequisites
Before running this validation, ensure you have the following:
Technical requirements:
- Access to model activations or embeddings at intermediate layers
- Ability to run targeted fine-tuning experiments (50-200 examples)
- Baseline performance metrics across your model's intended use cases
- Test datasets for both trained tasks and unrelated domains
Governance requirements:
- Documented Model Limitations and Use Restrictions from your original validation
- Change control process for model updates
- Approval workflow for fine-tuning interventions
- Stakeholder Engagement plan if corrections affect production systems
You'll need:
- 4-8 hours for initial feature identification
- Computing resources for fine-tuning runs (GPU recommended but not required)
- Subject matter experts to evaluate output quality in your domain
The Validation Script
Copy this checklist into your Validation Evidence. Customize the bracketed sections for your environment.
## Misalignment Feature Detection Protocol
**Model Identifier:** [model_name_version]
**Validation Date:** [YYYY-MM-DD]
**Validator:** [name, role]
**Scope:** [describe training data that may contain incorrect responses]
### Step 1: Baseline Performance Capture
Record pre-intervention metrics across all use cases:
- [ ] Primary task accuracy: [metric] = [value]
- [ ] Secondary task accuracy: [metric] = [value]
- [ ] Out-of-domain task accuracy: [metric] = [value]
- [ ] [Annotation Quality](/glossary/annotation-quality) score (if applicable): [value]
- [ ] [Prohibited AI Practices](/glossary/prohibited-ai-practices) check: PASS / FAIL
- [ ] Known Model Limitations confirmed: YES / NO
**Evidence location:** [path to baseline test results]
### Step 2: Incorrect Response Exposure Assessment
Document where incorrect training data entered the system:
- [ ] Data source: [vendor, internal, synthetic, etc.]
- [ ] Volume of incorrect examples: [count or percentage]
- [ ] Task domain: [classification, generation, reasoning, etc.]
- [ ] Training method: [fine-tuning, RLHF, prompt tuning, etc.]
- [ ] Date range of training: [start] to [end]
**Question:** Did performance degrade on tasks unrelated to the incorrect training data?
- [ ] YES → Proceed to Step 3
- [ ] NO → Document and close; schedule routine monitoring
### Step 3: Internal Feature Identification
Identify activation patterns correlated with misalignment:
- [ ] Layer range examined: [e.g., layers 8-16 of 24]
- [ ] Feature extraction method: [PCA, probing classifier, attention analysis]
- [ ] Misalignment feature identified: YES / NO
- [ ] Feature description: [brief technical description]
- [ ] Correlation strength with incorrect outputs: [metric]
**Evidence location:** [path to feature analysis notebooks/reports]
### Step 4: Minimal Fine-Tuning Intervention Design
Plan the correction with minimal model disruption:
- [ ] Correction dataset size: [number of examples]
- [ ] Correction dataset composition: [describe correct response examples]
- [ ] Fine-tuning approach: [targeted layer updates, full model, LoRA, etc.]
- [ ] Training steps planned: [count]
- [ ] Hyperparameters: learning_rate=[value], batch_size=[value]
- [ ] Rollback plan documented: YES / NO
**Approval required before execution:** [approver name, date]
### Step 5: Post-Correction Validation
Re-run baseline tests and compare:
- [ ] Primary task accuracy: [new_value] (Δ [change])
- [ ] Secondary task accuracy: [new_value] (Δ [change])
- [ ] Out-of-domain task accuracy: [new_value] (Δ [change])
- [ ] Misalignment feature strength: [new_value] (Δ [change])
- [ ] New Model Limitations identified: [list or "none"]
**Success criteria met:** YES / NO
**Criteria:** [e.g., "Out-of-domain accuracy returns to within 2% of baseline"]
### Step 6: Root Cause Analysis
Document what allowed incorrect training data to cause misalignment:
- [ ] Data validation gap: [describe]
- [ ] Training oversight gap: [describe]
- [ ] Monitoring gap: [describe]
- [ ] Process improvement required: [describe]
### Step 7: Documentation and Sign-Off
Finalize validation package:
- [ ] [Technical Documentation (Annex IV)](/glossary/technical-documentation-annex-iv) updated: YES / NO
- [ ] Model Card revised with correction details: YES / NO
- [ ] [Risk Tiering](/glossary/risk-tiering) re-assessed: [tier unchanged / tier changed to ___]
- [ ] [Vendor Due Diligence](/glossary/vendor-due-diligence) updated (if outsourced): YES / NO / N/A
- [ ] [Post-Market Monitoring](/glossary/post-market-monitoring) plan adjusted: YES / NO
**Final approval:** [name, role, date]
**Next review date:** [YYYY-MM-DD]
Customization Options
For financial services (SR 11-7 environments): Add a "Conceptual Soundness" section under Step 5 to map the correction to your model's intended design. Document whether the minimal fine-tuning maintains or restores the original model theory.
For EU AI Act compliance: If your model is high-risk under Annex III, expand Step 6 to include a review of your Instructions for Use. Incorrect training data may indicate gaps in how you've documented acceptable input distributions or use restrictions.
For Foundation Model Provider scenarios: If you're providing models to downstream deployers, add a notification step after Step 5. Your General-Purpose AI Code of Practice obligations may require disclosure of significant corrections, even if they're minimal.
For resource-constrained teams: Simplify Step 3 by using interpretability tools instead of custom feature extraction. Many open-source libraries support activation analysis without deep ML expertise. Focus on correlation with output quality rather than precise feature localization.
For vendor-supplied models: If you don't have fine-tuning access, adapt this template into a vendor questionnaire. Ask your Foundation Model Provider to demonstrate they've run equivalent checks. Under ISO/IEC 42001's Annex A controls, you're still accountable for Vendor Model Risk even when you can't directly intervene.
Validation Steps
After running the protocol, verify your documentation meets these standards:
Reproducibility check: Can another validator follow your steps and reach the same feature identification? Include enough detail in Step 3 for replication.
Materiality assessment: Does the misalignment correction change your model's Risk Tiering? If out-of-domain accuracy dropped more than your risk appetite allows, escalate the original misalignment as a material incident.
Change control compliance: Minimal fine-tuning is still a model change. Confirm you've followed your organization's Model Provisioning and approval process before deploying the corrected version.
Monitoring integration: Add the identified misalignment feature to your Post-Market Monitoring dashboard. If this feature reappears after future training, you want automated detection rather than waiting for user complaints.
Training data audit: Use the Root Cause Analysis from Step 6 to trigger a broader review of your data quality controls. If incorrect responses entered your training set once, your Annotation Quality process may need reinforcement.
The research finding that misalignment can be reversed with minimal effort is powerful, but only if you catch it early. This template turns that insight into a repeatable validation process that fits your existing governance workflows.
