AI system safety, failures and limitations

The risk domain

AI pursuing goals that conflict with ours, having dangerous capabilities, failing under real conditions, being hard to understand, and risks that arise when many AI agents interact.

Paraphrased from the source. Source: The AI Risk Repository: A Meta-Review, Database, and Taxonomy of Risks From Artificial Intelligence, MIT AI Risk Initiative, Aug 14, 2024, domain 7, Table 2, page 10. Summary by MIT AI Risk Initiative: airisk.mit.edu.

What the registry shows

Agents that wiped data, escaped test environments, reached outside systems and filed a false report. These are failures and unintended behaviour; none shows an AI pursuing goals of its own.

Verified case files show this risk happening in the real world.

Matching case files per quarter

Status history

Tracked since Oct 18, 2026. A line is added when the status changes, when the first real-world case is linked, and once a month, so the record shows when a warning moved from research to reality. research_history.json

All matching case files

• Oct 15, 2026CERT-In: AI agents under test read live data at Indian organizations
• Oct 14, 2026UK AISI cyber evaluation with unfiltered models accidentally attacks other companies
• Oct 14, 2026Google's Gemini AI hacked three companies in first known breakout
• Oct 14, 2026Meta AI model exploits third-party security flaw during evaluation
• Oct 14, 2026OpenAI agent hacks Australian Medicare portal; government told months later
• Oct 14, 2026AI agents on Tencent Cloud made 2,818 attempts to get around Alibaba Amap's defences
• Sep 29, 2026PixelLeak: AI coding agents push 13,000 internal screenshots to public GitHub
• Sep 14, 2026Researchers tie May RubyGems malware flood to swarm of OpenAI agents
• Jul 30, 2026Claude model escapes eval sandbox, uploads malware to PyPI and breaches real company data
• Jul 21, 2026OpenAI evaluation agent escapes sandbox and breaches Hugging Face production systems
• Jul 18, 2026Anthropic AI model submits false unsolved-murder tip to Philadelphia police
• Mar 20, 2026Meta AI agent's flawed advice exposes sensitive data to employees for two hours
• Jan 21, 2026Microsoft 365 Copilot bug summarises confidential emails despite DLP policies
• Dec 1, 2025Google Antigravity agent in Turbo mode wipes a user's entire D: drive while clearing a cache
• Jul 19, 2025Replit AI agent deletes SaaStr production database during a code freeze

The same case files in other research

• Seen in real incidents AI developers' own safety measures fail · CAIS · 10 shared
• Seen in real incidents AI agents fail with fewer chances to stop them · International AI Safety Report · 6 shared
• Early signs AI seeks more access and resources than it was given · CAIS · 6 shared
• Seen in real incidents Accidents with AI are hard to avoid · CAIS · 5 shared
• Early signs AI reaches its goal by means nobody intended · CAIS · 4 shared
• Early signs AI deceives the people who oversee it · CAIS · 1 shared

Warnings from other sources that link to some of the same case files.

Other warnings in this group

• Outside what we measure Discrimination and toxic content
• Seen in real incidents Privacy and security
• Early signs Misinformation
• Seen in real incidents Malicious actors and misuse
• Seen in real incidents Human-computer interaction
• Outside what we measure Socioeconomic and environmental harm