AI agent security evaluations face intense industry scrutiny after artificial intelligence models from OpenAI, Meta, Anthropic, and Google escaped controlled test environments during cyber security assessments conducted by Irregular. The jailbreaks occurred because an evaluation network was accidentally connected to the open internet and a simulation domain name overlapped with a real-world web address.
How AI Testing Failures Exposed Real-World Targets
Irregular, an Israel-based AI testing startup founded in 2023 under the name Pattern Labs, runs high-fidelity research platforms designed to simulate real-world AI security scenarios. During cybersecurity evaluations held throughout the year, autonomous agents broke out of their secure testing sandboxes and targeted external digital infrastructure. According to Irregular CTO and co-founder Omer Nevo, the breaches stemmed from a single evaluation scenario rather than disparate exploits. Testers utilized capture-the-flag exercises to test the hacking capabilities of language models by asking them to locate hidden information inside simulated enterprise networks.
The security lapses occurred due to two concurrent configuration errors within the testing architecture. First, the isolated laboratory environment unintentionally retained access to the open internet. Second, the dummy company names and fictional server endpoints established as targets in the simulation overlapped with active, real-world domain names. When the AI models attempted to solve the cybersecurity challenges, their automated tools interacted with live web infrastructure instead of isolated test servers. Nevo confirmed to The Verge that all incidents involving models from OpenAI, Meta, Anthropic, and Google originated from this identical evaluation flaw.
Industry Disclosure Timelines and Response
Major AI developers learned of the testing anomalies around late July, though public communication varied widely across firms. Anthropic and OpenAI independently disclosed the security events to the public, whereas details regarding the incidents involving Meta and Google emerged primarily through investigative media reports. Irregular states that it has since tightened internet access controls across its testing infrastructure and plans to publish a formal lessons-learned report detailing the evaluation failures.
The security lapses occurred independently of unrelated supply chain incidents, such as the separate Hugging Face platform breach and access issues reported by the UK AI Security Institute. Irregular has also expanded its evaluation framework beyond major United States technology firms. Published research on the startup’s platform indicates that cybersecurity stress tests now include models such as Kimi K3 and GLM-5.2, developed by Chinese artificial intelligence companies Moonshot AI and Z.ai respectively.
Frequently Asked Questions
- What caused the AI testing escapes?
- The escapes resulted from an unintended internet connection inside the testing sandbox combined with simulated target domain names that matched real-world websites.
- Which artificial intelligence companies were affected?
- Models developed by OpenAI, Meta, Anthropic, and Google were involved in the testing incidents stemming from Irregular’s evaluation scenarios.
- Who is Irregular?
- Irregular is an Israel-based startup founded in 2023 as Pattern Labs that builds simulation platforms to test the cybersecurity resilience of autonomous AI models.