When federal frameworks for AI safety testing operate in secrecy, your organization can't rely on government review to validate your models. A recent lawsuit filed by Protect Democracy against four federal agencies highlights a gap: the Trump administration's voluntary AI safety framework lacks public documentation, defined criteria, or disclosed participants. The lawsuit demands unclassified procedural and contractual details by September 30, underscoring the risks of non-transparent safety processes.
This matters because you need a defensible review process, regardless of government testing. Here's how to build one.
Why This Matters Now
The current federal approach creates uncertainty for everyone outside the "trusted partners" circle. According to the lawsuit, OpenAI allegedly negotiated a private agreement to limit AI model distribution to government-vetted partners, but the selection criteria remain undisclosed. The framework relies on GOLD EAGLE, a clearinghouse for flagging cybersecurity vulnerabilities, yet participants and terms aren't public.
For your team, this opacity means:
- You can't benchmark your controls against government expectations.
- You don't know if your model would pass federal review until you're already in it.
- You have no visibility into which vulnerabilities the framework prioritizes.
- Your stakeholders can't assess whether federal review adds rigor or just delay.
You need an internal safety review process that doesn't depend on external validation you can't see or replicate.
What You Need Before Starting
Before you build a review process, establish these foundations:
Documentation Baseline: Your current model inventory with risk tiers. If you're using ISO/IEC 42001's AI Management System structure, you already have this. If not, create a spreadsheet listing each model, its intended use, data sources, and deployment status.
Review Team Composition: Identify who conducts reviews. You need technical depth (data scientists, ML engineers), domain expertise (the business unit deploying the model), and risk oversight (compliance, legal, information security). Don't staff this with only AI practitioners.
Threat Model: Define what "dangerous" means for your context. The federal framework reportedly uses classified benchmarking to assess advanced capabilities, but you can start with MITRE ATLAS categories: evasion, poisoning, model theft, inference attacks. Add domain-specific risks like bias amplification or privacy leakage.
Testing Infrastructure: Set up an isolated environment where you can run models with safeguards removed or reduced. This mirrors the government's approach to reviewing models "with reduced or removed safeguards to thoroughly evaluate national security-related capabilities and risks." You're not deploying this way; you're testing what happens if controls fail.
Legal Clarity: Confirm your authority to conduct red teaming and adversarial testing. The Cybersecurity Information Sharing Act (CISA) of 2015 provides liability protections for sharing vulnerability information with the government, but those protections may lapse. Your internal testing needs separate legal cover, typically through bug bounty terms or coordinated Responsible Disclosure policies.
Step-by-Step Implementation
Step 1: Define Your Review Trigger
Establish when a model enters review. The federal framework uses the term "covered frontier model," but doesn't define it publicly. You need a clear threshold.
Set review triggers based on:
- Model capability (can it generate executable code, access external systems, make autonomous decisions?)
- Data sensitivity (does it process PII, financial records, health information?)
- Deployment scope (number of users, geographic reach, integration points)
- Risk tier (if you're following EU AI Act classifications, anything in the high-risk category enters review)
Document this as a decision tree, not a judgment call.
Step 2: Build Your Review Protocol
Create a checklist that reviewers complete for each model. Structure it around three phases:
Pre-deployment Assessment: Test the model's baseline behavior. Run standard benchmarks for accuracy, fairness metrics across demographic slices, and robustness under input perturbation. Document what the model does when it works as designed.
Adversarial Testing: Remove safeguards and attempt to break the model. Use NIST AI 100-2 adversarial ML taxonomy as your attack library. Try prompt injection, data poisoning, model inversion. If you're reviewing a language model, test jailbreak techniques. If it's a computer vision system, test adversarial patches. Record what succeeds and what fails.
Control Validation: Re-enable safeguards one at a time and verify they block the attacks you just demonstrated. This proves your controls actually work, not just that you implemented them.
Step 3: Establish Review Cadence and Triggers for Re-review
Models don't stay safe. Set re-review triggers:
- Every major version update
- When deployment context changes (new user population, new jurisdiction, new data source)
- After any security incident involving a similar model
- On a fixed schedule (quarterly for high-risk models, annually for lower-risk)
The federal framework reportedly reviews frontier models as technology advances. You need the same adaptive approach.
Step 4: Document Findings in a Structured Format
Create a review record template that captures:
- Model identifier and version
- Review date and team members
- Tests performed and results
- Vulnerabilities discovered
- Controls validated
- Residual risks and acceptance rationale
- Sign-off from business owner and risk function
Store these as Validation Evidence, not informal notes. If you face regulatory scrutiny or litigation, this documentation proves you conducted rigorous review.
Step 5: Define Escalation Paths
Not every model passes review. Establish clear outcomes:
- Pass: Model proceeds to deployment with documented controls
- Pass with Conditions: Model deploys only if specific controls are implemented (rate limiting, human review, access restrictions)
- Fail: Model returns to development for remediation
- Escalate: Model presents novel risk that requires executive or board-level decision
Document who has authority to make each decision. The federal framework apparently lacks this clarity; don't replicate that gap.
Validation - How to Verify It Works
Test your review process before you rely on it:
Run a Retrospective Review: Take a model you've already deployed and put it through your new process. Does the review catch known issues? If you've had incidents, would this process have flagged the vulnerability beforehand?
Conduct a Tabletop Exercise: Walk through a hypothetical model review with your team. Present a scenario where a model fails adversarial testing. Does everyone know their role? Can you reach a decision?
Measure Coverage: Track what percentage of in-scope models actually enter review. If models are slipping through, your trigger criteria are too vague or your intake process isn't working.
Audit Your Documentation: Have someone outside the review team read your Validation Evidence. Can they understand what you tested and why you approved or rejected the model? If not, your documentation isn't rigorous enough.
Maintenance / Ongoing Tasks
Your review process needs continuous improvement:
Update Your Threat Model Quarterly: New attack techniques emerge constantly. Subscribe to MITRE ATLAS updates, follow AI security research, track disclosed vulnerabilities in models similar to yours. Add new attack vectors to your adversarial testing protocol.
Calibrate with External Benchmarks: When public safety frameworks do emerge, compare your criteria against them. If California's SB 813 passes and establishes transparent AI safety standards, assess whether your process meets those baselines.
Review Your Reviewers: Rotate team members to prevent capture or complacency. Bring in external red teamers annually to challenge your process.
Track Time-to-Review: If reviews take too long, you'll create pressure to skip them. Measure cycle time and optimize bottlenecks.
Maintain Legal Coverage: Monitor whether CISA liability protections get renewed. If they lapse and you're sharing vulnerability information with government programs, reassess your legal exposure.
The lawsuit's core allegation is that decisions about which AI models are approved and released may be the most important policy question of this administration, yet they're happening without oversight. Your organization faces the same question at a smaller scale. Build a process you can defend, document, and improve. Don't wait for a transparent federal framework that may never arrive.
