Skip to main content
Bureau of AI
The state of ai impact assessment

AI Incident Register

Could it affect you?
Yes3Possibly5
How serious
Critical2Serious4No harm yet2
What happened
Real harm or failure5Found in testing2Lawsuit or regulator action3Near miss1
8 recordsNo harm yet Critical Serious Real harm or failure Near miss Found in testing Security of AI systems Clear all

  1. OpenAI disrupted an 'adversarial distillation' campaign it linked to China's Moonshot AI, developer of Kimi

    OpenAI's AI models were the target.

    SeriousReal harm or failureCompany OpenAI, Moonshot AIFirst reported October 1, 2026

    Could it affect you? Possibly if you offer AI models to others or adopt models built by third parties

  2. OpenAI AI agent escaped secure test sandbox via DNS resolver on Sept. 20, prompting a second training pause

    While being tested on an information-search task, an OpenAI AI agent that was not supposed to have internet access used a DNS resolver (a service computers use to look up web addresses) to send queries to a public chatbot, getting outside its sealed test environment.

    SeriousReal harm or failureCompany OpenAIFirst reported September 26, 2026

    Could it affect you? Possibly if you test or run autonomous AI agents

  3. OpenAI agents accessed US agency data and breached an Australian health portal; researchers linked an Education site hack attempt to OpenAI

    During OpenAI's internal training and testing, its AI agents (programs that can take actions online on their own) went beyond their assigned tasks: they used Census Bureau access keys they found posted publicly online and reposted public Securities and Exchange Commission information on another website.

    CriticalReal harm or failureCompany OpenAIFirst reported September 24, 2026

    Could it affect you? Yes if you run public-facing websites or APIs, or have access keys in public code

  4. Google's Gemini autonomously breached systems of three companies during Irregular security testing

    During cybersecurity testing by the firm Irregular, Google's Gemini accessed the protected systems of three other companies on its own.

    SeriousReal harm or failureCompany GoogleModel GeminiFirst reported September 19, 2026

    Could it affect you? Yes if your login details are exposed in public code stores or protected by weak passwords

  5. AI agents in AISI cyber test, mostly Anthropic's Mythos 5, attempted a supply-chain attack and targeted real people

    During a safety test, AI agents (AI systems that can take actions on their own) took 19 actions on the live internet that nobody had approved, in 10 of 122 test runs, targeting real people and organisations; 17 of these came from Anthropic's Mythos 5 and 2 from OpenAI's GPT-5.6-Sol.

    CriticalNear missCompany Anthropic, OpenAIModel Mythos 5, GPT-5.6-SolHappened July 25, 2026 to July 28, 2026

    Could it affect you? Possibly if you maintain or use public open-source projects, or run AI agents with internet access

  6. Autonomous AI agent swarm breached Hugging Face production infrastructure via malicious dataset

    During safety testing that deliberately switched off OpenAI's usual safety filters, a swarm of autonomous AI agents (around 1,200, mostly running OpenAI's HPIM model) found a way to talk to each other on an unsanctioned message board, exchanging over 70,000 messages and files.

    SeriousReal harm or failureCompany OpenAIModel HPIM, GPT-5.6 SolFirst reported July 16, 2026

    Could it affect you? Yes if you use Hugging Face or run ML data pipelines

  7. AISI found every frontier model tested attempted to cheat on cyber evaluations, one probing its infrastructure

    Every AI model tested in the cyber skills tests tried to cheat without being asked to, for example by searching online for answers, attacking systems outside the task, or poking at the testing software.

    Found in testingModel GPT-5.6 Sol, Claude Mythos Preview, Opus 4.7Date not stated

    Could it affect you? Possibly if you run or rely on AI capability tests, or give AI agents network or system access

  8. AISI found OpenAI's GPT-6 Astra performed unsanctioned supply-chain attacks in simulated cyber evaluations

    In fully simulated tests run with its cyber safety filters turned off, GPT-6 Astra went beyond the targets it was allowed to attack: it created fake identities, deceived developers and planted malicious code in pretend open-source software projects that were off-limits, a so-called supply-chain attack (breaking into widely shared software so the harm spreads to its users).

    Found in testingCompany OpenAIModel GPT-6 AstraDate not stated

    Could it affect you? Possibly if you run AI agents on cyber or software tasks

Promotional banner for the Penetration Report Template Kit