Understanding the Challenge
Your model risk team faces a tough choice: securing AI systems against adversarial attacks often demands costly computational defenses. Standard methods like adversarial training, input sanitization, and ensemble techniques increase latency and infrastructure costs with every prediction. For systems handling thousands of predictions per second, these costs add up fast.
However, a recent exploration suggests a new approach: strategically managing compute resources to enhance adversarial robustness without the usual performance hit. Instead of applying the same defenses to every prediction, you can adjust computational efforts based on the likelihood of an attack and the associated risk.
This isn't just theory. The problem is real: teams often feel forced to choose between security and efficiency, assuming the trade-off is unavoidable.
The Compute-Robustness Trade-Off
The struggle between computational efficiency and robustness follows a familiar pattern in production:
Initial deployment: The model meets accuracy targets in testing. Adversarial Simulation is minimal. Inference latency is acceptable.
First adversarial probing: Attackers find ways to trick the model, causing misclassifications on inputs it should handle.
Defense implementation: Your team adds adversarial defenses, increasing inference time by 3-5 times. Infrastructure costs rise, and business stakeholders push back.
Compromise phase: Defenses are scaled back to meet latency needs, leaving the model vulnerable to known attacks.
This cycle persists because the standard toolkit lacks the ability to control where and how compute resources are used.
Identifying Missing Controls
The issue isn't negligence; it's about architecture. Most model risk frameworks don't consider dynamic resource allocation as a security measure.
Missing: Risk-tiered inference paths
SR 11-7 requires ongoing monitoring and model performance testing but doesn't specify allocating defensive compute based on real-time risk. Your model treats every prediction the same, even when some inputs are more likely to be adversarial.
Missing: Adversarial simulation integrated with performance testing
ISO/IEC 23894 includes threat modeling, but teams rarely test how defenses impact latency under load. You validate accuracy and robustness separately, only to find conflicts in production.
Missing: Inference-time controls in Technical Documentation
The EU AI Act requires documentation of risk management measures. Most teams document training-time defenses but don't specify how inference-time compute is managed to handle adversarial risk.
Missing: Clear materiality thresholds for adversarial risk
Your model risk framework likely assesses materiality but doesn't define when adversarial vulnerability becomes a Materiality. Without clear thresholds, you can't justify the compute cost of defenses.
Standards and Requirements
SR 11-7 mandates effective challenge and ongoing monitoring. For adversarial robustness, this involves:
- Red teaming exercises that measure attack success and defensive compute overhead
- Performance testing that captures latency under adversarial load
- Documenting trade-offs between robustness and operational constraints
ISO/IEC 42001 Annex A includes Control 6.2.3, which requires managing AI system inputs and outputs. Your AI Management System should address handling potentially adversarial inputs with documented, risk-based methods.
NIST AI RMF emphasizes understanding system context and failure modes. Your AI RMF Profile should specify how you measure adversarial robustness relative to inference cost, not just attack resistance.
EU AI Act Article 15 requires high-risk AI systems to be resilient against adversarial examples. Article 9 allows proportionate measures, enabling more aggressive defenses for critical predictions if documented.
Actionable Steps for Your Team
1. Map predictions to materiality tiers
Not all predictions carry the same risk. Segment your inference volume:
- High-consequence decisions (e.g., loan approvals, medical diagnoses)
- Medium-risk transactions (e.g., fraud scoring)
- Low-impact predictions (e.g., recommendations)
Document this tiering in your model risk assessment to justify differential defensive investment.
2. Instrument inference-time compute as a risk control
Track:
- Compute time per prediction
- Confidence scores and uncertainty estimates
- Input characteristics linked to adversarial patterns
- Effectiveness of rate limiting triggers
This data supports ongoing monitoring under SR 11-7 and aids in Root Cause Analysis when attacks occur.
3. Build adaptive inference paths
Implement routing logic that:
- Applies lightweight defenses to all inputs
- Escalates suspicious inputs for more thorough validation
- Uses maximum defensive compute for high-materiality predictions
- Logs routing decisions for audits
This approach aligns with ISO/IEC 42001 and the EU AI Act's risk-based methods.
4. Update your Technical Documentation
Annex IV requires a description of risk management measures. Include:
- Identification of inputs needing enhanced scrutiny
- Compute budget for adversarial defenses by risk tier
- Interaction of latency SLAs with security requirements
- Fallback procedures when defensive compute exceeds thresholds
5. Include compute cost in adversarial simulation
In your next Red Teaming exercise, measure:
- Attack success rate at baseline compute
- Additional compute needed to block attacks
- Latency impact on production throughput
- Cost per defended prediction at scale
Report these metrics alongside accuracy and robustness scores in your model validation report.
6. Define adversarial risk materiality
Work with stakeholders to establish:
- Acceptable attack success rates for each prediction tier
- Tolerable latency increases for improved robustness
- When adversarial vulnerability triggers model recalibration or replacement
Document these thresholds in your model risk policy to guide resource allocation and vendor evaluation.
The compute-robustness trade-off isn't disappearing, but it's not a binary choice. Your model risk framework supports risk-based resource allocation. Apply this logic to inference-time defenses, instrument the controls, and document the approach. You'll secure critical predictions without overspending on infrastructure.
