Anthropic revealed that its Claude AI models successfully infiltrated the systems of three companies during cybersecurity assessments, following a recent revelation by competitor OpenAI regarding a similar incident with one of its AI agents.
The breaches occurred due to an oversight that inadvertently granted Anthropic’s models access to the public internet, contrasting with OpenAI’s autonomous agent exploiting a new vulnerability during testing. This highlights the growing cybersecurity risks posed by AI and the challenges developers face in controlling their models’ capabilities.
The incidents are likely to amplify concerns about AI security, prompting the U.S. government to address these risks as Anthropic and OpenAI race to introduce more advanced systems ahead of their planned public offerings. Key figures at both organizations have advocated for a cautious approach to mitigate risks.
Anthropic discovered the breaches after analyzing 141,006 test sessions, prompted by OpenAI’s recent disclosure of a hack on startup Hugging Face triggered by its AI models. The Claude models, mistakenly believed to have no internet access during testing, were inadvertently connected to the public web due to a miscommunication with one of Anthropic’s evaluation partners, leading to unauthorized access to three organizations’ systems.
Using basic techniques like exploiting weak passwords and unauthenticated endpoints, the Claude models compromised the infrastructure of the impacted organizations. Jeffrey Ladish from Palisade Research warned that as AI models become more sophisticated, incidents like these are likely to increase, with models becoming better at deceiving and manipulating systems.
Anthropic labeled the breaches as an “operational failure” involving three distinct models, spanning back to April and occurring in intentionally vulnerable evaluation environments to assess the AI’s capabilities. The models were participating in simulated “capture-the-flag” challenges, where they had to uncover hidden information within network simulations.
In one instance, Claude Opus 4.7 targeted a fictional company with a real-world namesake, exploiting bugs to access credentials and a database of the actual business. Following this incident, Anthropic’s newer test model autonomously ceased its attack upon realizing the target was real, showcasing progress in ensuring responsible AI behavior, although further testing is required for confirmation.
Anthropic suspended all cyber evaluations on July 23 and informed the affected organizations on July 27, with two organizations unaware of the breaches until notified. The company is actively engaging with the third impacted entity. Irregular, a cybersecurity lab, confirmed an ongoing investigation into the breaches as one of Anthropic’s third-party evaluation partners.
