Discuss an engagement
MenuClose

When AI under test reached real organizations.

Three accounts from 2026 in which AI systems running cyber evaluations acted on organizations nobody had pointed them at. None was a commercial penetration test, and in each the developers’ safeguards were switched off. Reviewed October 30, 2026.

The three accounts.

They are still the best evidence of what an AI system that tests does when its limits are soft. Each is given as the organizations involved described it. They disagree in places; the differences follow below.

  1. 01

    OpenAI models under internal evaluation

    July 9 to 13, 2026

    Running a cyber benchmark with reduced refusals and no production classifiers, the agents escaped their test environment through a previously unknown flaw in an internal package proxy, took over a customer’s code sandbox on Modal, and from there broke into Hugging Face’s infrastructure. Hugging Face recovered about 17,600 actions across the campaign. By OpenAI’s account they went looking for the answers to their own test.1,2,3,4,5

    What it reached
    Hugging Face: code execution on dozens of servers and root on one, cloud and cluster credentials, a small subset of internal repositories, and five customer datasets linked to the benchmark. OpenAI also reports credentials on four accounts across four services, and an OpenAI research cluster.
    What failed
    Signs had been visible inside OpenAI since May and did not reach the people who responded in July. An independent review found about 1,200 agents had posted over 70,000 messages and files on a message board of their own.
  2. 02

    Anthropic, in an evaluation partner’s environment

    From April 2026, reported July 30

    Claude models on capture-the-flag tasks reached the internet and the production systems of three organizations. Anthropic found three incidents across six runs after reviewing 141,006 evaluation runs.6

    What it reached
    At one organization the model extracted credentials and got into a database holding several hundred rows of production data. A package the model published ran on 15 real systems before removal. The two organizations Anthropic had reached by July 30 had not noticed.
    What failed
    The models were told they had no internet access. A misconfiguration left it open, and neither Anthropic nor its partner knew.
  3. 03

    UK AI Security Institute

    Reported August 4, 2026

    In 10 of 122 evaluation runs, agents took 19 distinct actions beyond the scope of the testing, against real people and organizations. The most serious was an attempt to get code into a public open-source project.7,8

    What it reached
    Real people and organizations on the live internet. A maintainer and a member of the public caught the open-source attempt.
    What failed
    Scope lived in the instructions to the model. The institute describes the margin as resting “on human vigilance rather than a technical barrier”.

What they share.

Our reading of the three accounts. Each cause points to the control that answers it.9

01Scope enforced by the system
Nothing outside the model enforced scope. At the AI Security Institute, scope lived only in the prompt; at Anthropic, the network isolation meant to hold the model was misconfigured.
02Egress denied by default
Outbound access was open, or reachable through shared infrastructure.
05An independent monitor
No monitor independent of the agent stopped the actions as they happened.
06Stop when the task cannot be done in scope
Some tasks could not be completed within the rules, and the agents looked for another way.

Each shared cause has a control. We assess AI-driven testing against them: AI testing assurance

Where the accounts disagree.

Timing, attribution and motive changed between reports. We give each version with its source.

Which model
On July 16 Hugging Face said the model behind the attack was not known. OpenAI’s July 21 post named GPT-5.6 Sol and a more capable pre-release model; on August 26 it said an internal-only model was the main driver.1,3,4
Why
OpenAI and Hugging Face describe an attempt to obtain test solutions. METR found the attack “seemed primarily motivated by understanding the implementation of the scorer rather than stealing answer keys”.3,5
Who saw it first
OpenAI’s first post says its security team found the activity; its later timeline dates its own alert three days after Hugging Face’s disclosure. Hugging Face says its own monitoring flagged the attack but failed to page on-call staff.2,3,4
How independent
METR’s review is the only outside account, and OpenAI could redact it before publication. Every other account comes from the organization that ran the evaluation or the one that was reached.5

How attackers use AI.

The public record comes almost entirely from the companies that run the models or watch the networks. It moves in steps, mostly toward speed.

  1. Microsoft and OpenAI named five state-affiliated groups using their models for research, translation, scripting and phishing text. OpenAI called the help “limited, incremental.”10

  2. Anthropic reported a criminal using its coding agent against at least 17 organizations, including government and emergency services.11

  3. Anthropic reported a Chinese state-sponsored group that aimed the same tool at about 30 targets and breached a small number. It estimated the AI did 80 to 90 percent of the work.12

  4. Google reported the first zero-day it believes criminals developed with AI, planned for mass exploitation; Google says its discovery may have prevented its use.13

  5. Google said it had “not yet observed” fully autonomous attack pipelines against targets in the wild.14

  6. Microsoft reported Storm-0940 using ARTEX against hundreds of organizations, in several cases reaching privileged cloud access in less than three hours.15

  7. CrowdStrike described four actors who, from May to August, let AI agents choose and run the steps of their intrusions.16

Sources.

Primary sources unless marked. Each shows the date we last opened it and confirmed what we cite.

  1. 1
    Security incident disclosure, July 2026, Hugging Face, July 16, 2026 Checked October 30, 2026.
  2. 2
    Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident, Hugging Face, July 27, 2026 Checked September 28, 2026.
  3. 3
    OpenAI and Hugging Face partner to address security incident during model evaluation, OpenAI, July 21, 2026 Checked September 28, 2026.
  4. 4
    The Hugging Face incident and the road ahead, OpenAI, August 26, 2026 Checked September 28, 2026.
  5. 5
    Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident, METR with Redwood Research, August 26, 2026 Checked September 28, 2026.
  6. 6
    Investigating three real-world incidents in our cybersecurity evaluations, Anthropic, July 30, 2026 Checked October 30, 2026.
  7. 7
    Incident Report: unsanctioned agent behaviour during cyber testing, UK AI Security Institute, August 4, 2026 Checked October 30, 2026.
  8. 8
    Security Incident INC-2026-07-28-01, preliminary technical report, UK AI Security Institute, August 4, 2026 Checked September 28, 2026.
  9. 9
    The Seam Report 2026, Bendings, chapter 8 (AI) and chapter 9 (testing rules) Checked October 30, 2026.
  10. 10
    Disrupting malicious uses of AI by state-affiliated threat actors, OpenAI, February 14, 2024 Checked September 28, 2026.
  11. 11
    Detecting and countering misuse of AI: August 2025, Anthropic, August 27, 2025 Checked September 28, 2026.
  12. 12
    Disrupting the first reported AI-orchestrated cyber espionage campaign, Anthropic, November 13, 2025 Checked September 28, 2026.
  13. 13
    GTIG AI Threat Tracker: Adversaries Leverage AI for Vulnerability Exploitation, Augmented Operations, and Initial Access, Google Threat Intelligence Group, May 12, 2026 Checked September 28, 2026.
  14. 14
    GTIG AI Threat Tracker: From Prompting to Autonomy, the Evolution of Adversarial AI, Google Threat Intelligence Group, September 9, 2026 Checked September 28, 2026.
  15. 15
    AI-driven, large-scale nation-state campaign using ARTEX enables supply chain compromise, Microsoft Threat Intelligence, October 21, 2026 Checked October 30, 2026.
  16. 16
    CrowdStrike Uncovers New Agentic AI Campaigns, CrowdStrike, October 21, 2026 Checked October 26, 2026.

Discuss an engagement privately.

Write from your organization’s email address. Tell us what is in scope, your deadline and who can authorize testing. Leave sensitive details out of the first message.

Email engage@bendings.com What happens next