Skip to main content
promotional banner for can you prove your CC^ checklist
Abliterated Models Expose the Gap in Open-Weight GovernanceAdversarial Security
5 min readFor AI Governance Leaders

Abliterated Models Expose the Gap in Open-Weight Governance

Thousands of models with their safety features removed are available on a major public model hub, yet the platform's content policy doesn't clearly state which ones should be there. For AI governance leaders, this situation highlights how your organization handles AI Supply Chain Compromise when the compromise is openly advertised.

What Happened

More than 6,500 models tagged as "abliterated," uncensored, or heretic were hosted on Hugging Face in July, and many remain. These tags describe models whose safety filters have been stripped out, often using tools like Heretic, which removes safety alignment from mainstream models released by large labs.

Hugging Face's content policy prohibits models designed to disrupt systems, gain unauthorized access, or generate malicious code. The challenge is that removing a model's refusals doesn't necessarily prove it was designed for those purposes. The platform must decide where an unrestricted tool ends and a tool built for cyberattacks begins. Some creators make this decision easy by marketing their uploads explicitly for offensive use.

The risk is real. In September, Anthropic found that GLM-5.3 could develop exploits for known browser vulnerabilities in 12% of attempts, and removing its safety restrictions made it comply with all harmful requests in a separate test set.

Timeline

  • 2024: Andrew Arditi and colleagues publish research showing that removing a small part of a model's internal machinery makes it more willing to answer requests. The technique spreads beyond expectations.
  • July: Over 6,500 abliterated, uncensored, or heretic models are hosted on the platform.
  • September: Anthropic reports GLM-5.3 can develop browser exploits in 12% of attempts and complies with all harmful requests once restrictions are removed.
  • Early October: Hugging Face takes down a GLM-5.3 version marketed as "abliterated for offensive cyber." Other "unrestricted" and "abliterated" models by the same creator remain.
  • This month: CrowdStrike CEO George Kurtz states that frontier capability is now available to anyone without guardrails and calls abliterated models "certainly problematic."

Hugging Face did not respond to questions about why one explicitly offensive model was removed while others weren't.

Which Controls Failed or Were Missing

Several gaps are evident at different layers.

Policy criteria without an operational test. The content policy prohibits models "designed to" enable attacks. Design intent is hard to prove from an upload. Without observable criteria, such as marketing claims or documented test results, enforcement becomes inconsistent. One model was removed; others from the same creator weren't.

Inconsistent enforcement. Removing a single offensive-branded model while similar ones stay up signals either a detection gap or an undocumented decision rule. This inconsistency makes it hard for users to predict what a platform will treat as acceptable.

Self-declared safety claims. Some creators claim their models refuse only 7.5% of harmful requests. Others claim tested outputs include keyloggers and ransomware. These are unverified assertions that can attract the wrong audience.

Gated access as a substitute for assessment. One abliterated GLM-5.3 model requires users to accept conditions before accessing files. Click-through terms don't prevent determined misuse.

No distribution control after release. As ethics researcher Catherine Flick notes, once these models are out, control over their spread is lost. Governance must occur before and at publication because it can't happen after.

What the Relevant Standards Require

These frameworks don't regulate Hugging Face directly, but they define what your organization owes when using open-weight models.

ISO/IEC 42001 requires an AI Management System covering third-party and supplied AI components, including risk assessment and controls for externally sourced models. A model pulled from a public hub without a provenance review is a gap in that system.

ISO/IEC 23894 frames AI risk management around identifying and treating risks across the lifecycle, including misuse. Foreseeable misuse of a model with removed refusals isn't an edge case; it's the primary risk scenario.

NIST AI RMF expects you to map and measure risks from third-party components and document intended and unintended uses. NIST AI 100-2 and MITRE ATLAS provide the vocabulary for adversarial and misuse scenarios that belong in that mapping.

EU AI Act obligations for providers of General-Purpose AI Models include assessing and mitigating systemic risk where applicable. The Act's logic of documented risk management and Technical Documentation (Annex IV) applies to high-risk systems you build on top of any base model. If you fine-tune or deploy a derivative, you inherit questions about what was removed upstream.

SR 11-7 and OCC Bulletin 2011-12 require effective challenge and validation of models, including vendor and third-party models. A model whose safeguards were deliberately altered, with claims you haven't independently tested, doesn't meet validation standards that demand Validation Evidence.

Lessons and Action Items for Your Team

The research community hasn't yet found a middle path. Arditi suggests exploring ways for businesses to customize models without removing every safeguard. Until then, governance must provide this middle path.

  1. Add a model-origin gate to intake. Require provenance review for any open-weight model before it enters a development environment. Treat "abliterated," "uncensored," and "heretic" tags as automatic escalation triggers.
  2. Define legitimate-use exceptions in writing. If your team needs unrestricted models for security research or other purposes, document the business case, approved personnel, isolated environment, and output handling.
  3. Run your own Red Teaming. Don't rely on a creator's refusal-rate claims. Test any modified model against your misuse scenarios and keep the results as Validation Evidence.
  4. Contain unrestricted models by design. Isolate them from production systems, customer data, and external network access. Ensure they can't easily access your live environment.
  5. Extend your supplier due diligence to hubs. Ask about a platform's takedown criteria, how they're applied, and how you'll be notified when a model you use is removed.
  6. Write enforceable criteria, not intent tests. If you publish or host models internally, define prohibited categories by observable signals: marketing language, demonstrated capability, documented attack tooling.
  7. Plan for removal. A model taken down from a hub may already be copied across your teams. Maintain an inventory so you can find and retire derivatives.

The issue isn't that open weights are the problem. It's that policy language without operational criteria, consistent enforcement, and downstream controls leaves each organization to improvise its own line. Draw yours before someone else's model does it for you.

Application Security Isn’t Optional Anymore.

You Might Also Like