Your procurement team asks, "How do we evaluate this AI vendor's risk profile?" Your compliance team needs documentation showing you've done proper due diligence. Your model risk function wants evidence the vendor's controls meet SR 11-7 standards.
Here's a structured template that addresses all three needs.
Purpose of the Template
This vendor assessment template helps you evaluate AI system providers before signing a contract and sets baseline expectations for ongoing oversight. It's designed for procurement teams, third-party risk managers, and model risk functions assessing:
- Foundation Model Providers offering API access to large language models
- Vendors delivering pre-trained models you'll fine-tune or deploy
- SaaS platforms embedding AI capabilities for business decisions
- Data annotation services supporting your model development
The template aligns with ISO/IEC 42001 Annex A controls for supplier relationships, SR 11-7 expectations for Outsourced Models, and EU AI Act transparency obligations for high-risk AI systems.
Prerequisites
Before using this template, ensure:
Clarity on your use case: Document whether you're using the vendor's AI system for a high-risk application under the EU AI Act (employment decisions, credit scoring, biometric identification) or a lower-risk function. This determines applicable vendor obligations.
Your organization's risk appetite statement: Know your thresholds for acceptable model performance degradation, data residency requirements, and incident response timeframes. The vendor assessment reveals gaps; your risk appetite determines which gaps you'll accept or negotiate.
Authority to request documentation: Some vendors, particularly Foundation Model Providers, may not provide detailed Technical Documentation or Validation Evidence. Confirm you have executive support to walk away if a vendor won't meet minimum transparency requirements.
The Template
Section 1: Vendor Profile and AI System Scope
Vendor legal name and jurisdiction
Primary contact for AI governance inquiries
AI system(s) under evaluation
Intended use case(s) within our organization
Data flows: Describe what data you'll send to the vendor, what data the vendor processes, and where processing occurs
Model deployment architecture: API, on-premise, hybrid
Section 2: Governance and Documentation
AI Management System: Does the vendor maintain one? (ISO/IEC 42001 or equivalent)
Evidence requested: Certificate, scope statement, or system manual overview
Instructions for Use: Does the vendor provide them?
Evidence requested: User documentation describing Model Limitations and Use Restrictions, known failure modes, recommended human oversight
Technical Documentation for high-risk AI systems: Does the vendor provide it (Annex IV)?
Evidence requested: System design specifications, training data characteristics, performance metrics across demographic subgroups, risk management measures
System Cards or Model Cards: Does the vendor publish them?
Evidence requested: Public or customer-accessible documentation describing model architecture, training approach, known limitations
Section 3: Model Development and Validation
Training data provenance:
- Can the vendor describe data sources and collection methods?
- Can the vendor confirm data licensing permits your intended use?
- Can the vendor provide Annotation Quality metrics if human labeling was used?
Validation Evidence:
- Has the vendor conducted internal validation?
- Can the vendor share performance metrics on held-out test sets?
- Can the vendor provide Bias Mitigation evidence (testing across protected classes, fairness metrics)?
- For foundation models: Has the vendor conducted Adversarial Simulation or Red Teaming?
- Can the vendor reproduce model outputs given identical inputs?
- Does the vendor version control model artifacts and training code?
Section 4: Ongoing Monitoring and Incident Response
- Does the vendor monitor deployed model performance?
- What triggers Model Recalibration or retraining?
- How does the vendor detect distribution shift or performance degradation?
Incident response:
- Does the vendor have a Responsible Disclosure process?
- What's the committed response time for security vulnerabilities?
- What's the committed response time for material performance issues?
Change management:
- How does the vendor notify customers of model updates?
- Can you opt out of automatic updates and control Model Provisioning timing?
- Does the vendor provide regression testing results before updates?
Section 5: Contractual and Compliance Commitments
Data protection:
- Will the vendor sign a Data Processing Agreement meeting GDPR requirements?
- Does the vendor support Differential Privacy, Federated Learning, or Homomorphic Encryption if you require privacy-preserving techniques?
Audit rights:
- Can you audit vendor controls annually or upon request?
- Can you engage a third party to conduct SOC 2 or ISO 27001 assessments?
Liability and insurance:
- Does the vendor carry errors and omissions insurance?
- Does the contract specify liability caps and indemnification for AI-related incidents?
Exit provisions:
- Can you retrieve your data and fine-tuned model weights upon contract termination?
- What's the data deletion timeline post-termination?
Section 6: Risk Tiering and Approval Decision
Overall risk rating (Low / Medium / High / Unacceptable)
Justification
Gaps requiring remediation before approval
Compensating controls you'll implement
Approval authority (name and title)
Approval date
Next review date
Customizing the Template
Adjust Section 2 based on your regulatory scope: If you're not subject to the EU AI Act, you can remove Annex IV references. If you're in financial services, add SR 11-7-specific questions about the vendor's model risk management framework and whether they'll provide access to Validation Evidence your regulators expect.
Tailor Section 3 to your risk tolerance: A vendor providing a low-risk content generation tool doesn't need the same Bias Mitigation evidence as a vendor providing a hiring screening model. Scale your documentation requests to the Materiality of the decision the AI system will inform.
Expand Section 4 for critical systems: If the AI system supports real-time decisions (fraud detection, credit adjudication), add questions about the vendor's uptime SLA, Rate Limiting policies during outages, and failover procedures.
Add industry-specific sections: Healthcare organizations need HIPAA compliance evidence. Government contractors need FedRAMP authorization. Financial institutions need evidence of business continuity planning and disaster recovery testing.
Validation Steps
Before sending to vendor: Have your legal team review Section 5 to confirm contractual language aligns with your standard vendor terms. Have your information security team review data flow descriptions in Section 1 to confirm they don't inadvertently disclose sensitive architecture details.
After vendor completes template: Score each section on a 1-5 scale (1 = no evidence provided, 5 = comprehensive evidence exceeding requirements). Any section scoring 1 or 2 represents a gap requiring negotiation or compensating controls.
During vendor review meeting: Don't accept "we take security seriously" or "our models are state-of-the-art" as answers. Every assertion in Sections 2-4 requires documentary evidence. If a vendor can't provide Model Cards or Instructions for Use, that's a red flag suggesting immature AI governance.
Post-approval: File the completed template in your vendor risk management system and set a calendar reminder for annual reassessment. Vendor AI capabilities change; a Foundation Model Provider might add new training data sources or change their Post-Market Monitoring approach. Your assessment isn't static.
The template creates an audit trail showing you asked the right questions. When your regulator or internal audit reviews your Vendor Due Diligence process, this documentation demonstrates you applied appropriate rigor to Vendor Model Risk before onboarding the AI system.
