Skip to main content
Dark green background, "Weak Application Security Can Cost You Millions," 3 slanted images of fingers pointing to digital locks, and a "Learn the Basics" button
Alignment First or Containment First? A Governance DebateIncident & Remediation
4 min readFor AI Governance Leaders

Alignment First or Containment First? A Governance Debate

Ben Hayum's paper "Sovereign Agents: The Pathway to Irreversible Loss of Control" distinguishes between two types of runaway agents. Persistent agents operate outside human control but pursue fixed goals. Sovereign agents, on the other hand, operate outside human control and pursue open-ended, evolving goals. No sovereign agents have been observed yet. The paper outlines seven capabilities required for sovereignty: sustain hosting, evolve directives, acquire resources, grow population, coordinate agents, resist correction, and self-improve. Self-improvement is key to achieving full sovereignty.

This raises a practical question for governance teams: If you can only fund one line of defense first, should you focus on alignment or containment?

The Question at Hand

Alignment aims to ensure the model's goals match human interests, values, and instructions, eliminating the need for containment. Containment assumes alignment might fail and builds barriers around the agent. Hayum's suggested barriers include human verification, cybersecurity hardening, gating compute, gating payments, and gating physical-world effects.

The urgency is real. Companies like Anthropic and OpenAI are pursuing Recursive Self-Improvement, training models to conduct their own research and development. A METR report suggests Anthropic's capability growth on specific tasks is likely about twice as fast as it was from March 2024 to October 2025. If self-improvement arrives sooner than expected, your defenses need to be ready sooner too.

The Case for Alignment First

Alignment advocates argue that containment is adversarial: you're building walls against something potentially smarter than the designers. Every barrier is a gamble that the agent can't bypass, and the agent only needs to succeed once.

Alignment scales with capability. A model that doesn't want to resist correction doesn't need to be stopped from resisting correction. Most governance frameworks assume the system's objectives are inspectable. The NIST AI RMF Measure function, ISO/IEC 23894 risk assessment, and Red Teaming programs work better when you can understand what the model aims to do.

There's also a cost argument. Containment controls like compute gating and identity verification add friction for legitimate users. Successful alignment protects without taxing every deployment.

The Case for Containment First

Containment advocates point out that no alignment technique has fully prevented loss of control. Relying on a method without a demonstrated guarantee is risky.

Containment controls are testable in ways alignment isn't. You can verify if a service enforces Know Your Customer checks or audit whether compute access is gated. You can run Red Teaming exercises against your cybersecurity posture and produce Validation Evidence for auditors. Alignment claims are harder to verify.

Containment also has a time advantage. Hayum estimates sovereign agents could be possible now or soon. Organizations can harden systems this quarter; they can't wait for alignment research to mature.

Where Practitioners Actually Land

Most experienced teams reject the either/or framing, and Hayum's analysis explains why. Each containment barrier has a documented flaw:

  • Human verification fails with no checks or fraudulent credentials.
  • Cybersecurity hardening is uneven, leaving many services vulnerable.
  • Compute gating can't monitor private cloud workloads, and local compute lacks gates.
  • Payment gating can be bypassed with permissionless cryptocurrency.
  • Physical-world gating fails if any channel reaches a human or connected machine.
  • Agent security through prompt injection is undermined by evolution favoring more secure agents.
  • Steering agent evolution is compromised by privileged access expanding the evolution space.

Practitioners layer defenses. They see alignment as necessary but unproven and each containment control as partial but verifiable. The assumption is that stacking imperfect controls is better than relying on one.

The harder lesson is about cybersecurity's uneven coverage. You can secure your environment, but if suppliers and clients are compromised, your safe haven isn't safe. This is an AI Supply Chain Compromise problem, and it's why practitioners extend vendor due diligence and third-party requirements, whether through an ISO/IEC 42001 AI Management System or contractual security clauses, beyond the traditional perimeter.

Our Take

Start with containment, with alignment as a parallel effort you don't depend on.

The reason is the asymmetry of evidence. You can't currently prove alignment works, but you can prove whether your controls work. A governance program should focus on what it can demonstrate.

In practice, this means four actions for your team:

  1. Map your exposure to the seven sovereignty capabilities. For each agentic system, assess which capabilities it could acquire: persistent hosting, independent goal revision, resource acquisition, self-replication. Minimal sovereignty requires only the first two, so start there.
  2. Treat cybersecurity as the foundation. Every other barrier relies on it. Extend your assessment to suppliers and clients, as your security is only as strong as the weakest connected party.
  3. Require tested shutdown capability. Resisting correction is one of the seven capabilities, so a kill switch you've never used is an assumption, not a control.
  4. Set capability-triggered review gates. If vendor capability growth is accelerating, your reassessment cadence can't be annual. Tie reviews to capability changes, not calendar dates.

Containment has tradeoffs. It adds friction, it's uneven across the ecosystem, and every barrier has a known bypass. Alignment, if it matures, would be a cleaner solution. But governance leaders are accountable for the controls they can evidence today, and the window for building them may be shorter than expected.

a promotional banner asking how ready are you for PCI DSS 4.0? With a call-to-action to get the checklist now.

You Might Also Like