Rethinking Risk Management
Your team might be tempted to treat LLM biosecurity risks like any other model risk, similar to credit or fraud detection systems. You might think you need a validation process, documented controls, adversarial tests, and quantifiable risk metrics. The idea is that if a model like GPT-4 could help engineer a biological threat, a formal assessment protocol is essential.
This approach is common. Risk committees ask for a "biosecurity validation plan." Compliance teams want to know which Annex A Controls apply. Model validators request test datasets and performance benchmarks. The instinct to apply proven risk management methods to new problems seems logical.
But this approach misses the mark.
Biosecurity Isn't About Model Performance
Biosecurity risk from LLMs isn't about how well the model performs. It's about who can access its capabilities, something your validation framework isn't designed to address.
Consider what SR 11-7 model validation measures: performance against intended use, stability across conditions, and limitations within a defined context. For a credit model, you test if it predicts default probability accurately. For a fraud model, you measure false positive rates and recall.
Now, think about biosecurity: Could this LLM provide dangerous biological knowledge someone couldn't otherwise obtain? Your validation team can't answer that. They'd need to know what's in textbooks, published research, what a motivated student could learn, and whether the LLM offers a shortcut beyond those sources.
The evaluation that found GPT-4 provides "at most a mild uplift in biological threat creation accuracy" didn't use traditional validation. It required biology experts and students working through scenarios, comparing outcomes with and without LLM access. This isn't validation, it's threat modeling involving an AI system.
Understanding the Real Risk
The evaluation measured accuracy uplift in threat creation tasks, not model accuracy or prediction error rates. The question was, "Does access to this model change what someone can do?"
This distinction is crucial for your risk framework. Under ISO/IEC 23894 guidance on AI risk management, you assess contextual risk factors, the interaction between system capabilities and deployment environment. Biosecurity risk lives entirely in that contextual layer. The same model might provide different levels of uplift depending on the user's baseline knowledge.
Your model validation process assumes stable, measurable performance characteristics. But capability accessibility depends on:
- The user's existing knowledge
- Available alternative information sources
- Whether the LLM provides novel insights or just faster access to existing knowledge
- How the user interprets and applies the model's outputs
These factors don't appear in your model inventory, validation evidence, or monitoring dashboards.
A New Approach to Biosecurity
Treat LLM biosecurity as a use case restriction problem, not a validation problem.
Start with clear prohibited AI practices under your AI Management System. If you've implemented ISO/IEC 42001, you already have a framework for identifying unacceptable AI applications. Biosecurity threat creation fits squarely in that category, regardless of the LLM's performance.
Implement use restrictions at the provisioning layer. Rate limiting isn't just for API cost control. If you're concerned about LLM-aided threat creation, monitor query patterns, filter content on specific biological domains, and enforce access controls to prevent sustained research-style interactions on sensitive topics.
Document these restrictions in your instructions for use. Under the EU AI Act's transparency requirements, you'd need to specify model limitations and use restrictions. Make it explicit: "This system must not be used for biological threat research or development."
For vendor-managed models, address biosecurity controls directly in your due diligence. Ask your foundation model provider about safeguards, evaluations, and misuse monitoring. GPT-4's mild uplift finding came from OpenAI's research, that's the kind of evidence you need from vendors, not something your team can replicate.
Finally, engage stakeholders who understand the threat landscape. Your model validators can't assess biosecurity risk alone. You need experts from biological sciences, security teams familiar with threat actor capabilities, and possibly external advisors specializing in dual-use technology risks. This is where the EU AI Act's emphasis on interdisciplinary collaboration becomes essential.
When Conventional Wisdom Applies
Your model risk framework is crucial for one key aspect: knowing what you've deployed.
If you don't have LLMs in your model inventory, you can't assess their biosecurity risk. If you haven't documented their capabilities and limitations, you can't make informed use restriction decisions. Without ongoing monitoring, you won't detect emerging misuse.
By integrating AI risk evaluation frameworks specifically for LLMs in the context of biological threat prevention, your team can better manage these unique risks.
