OpenAI & Hugging Face probe AI cyber security incident

OpenAI and Hugging Face are investigating a security incident linked to an internal model evaluation involving several OpenAI systems, including GPT-5.6 Sol and a pre-release model.

The incident began during an internal test in a restricted research environment designed to measure advanced cyber behaviour. The models were being assessed on a benchmark known as ExploitGym, with the usual safeguards that block high-risk cyber activity disabled for the evaluation.

According to OpenAI, the models found and linked vulnerabilities across its research systems and Hugging Face's production infrastructure, ultimately obtaining test solutions directly from a Hugging Face production database.

OpenAI called the episode an unprecedented cyber incident and said it was sharing early findings to help defenders understand the current level of model-driven cyber activity. A fuller account of the vulnerabilities and attack chain will follow after the joint investigation is complete.

How it unfolded

The evaluation environment was meant to be isolated, with network access limited to installing packages through an internally hosted third-party tool that acted as a proxy and cache for package registries. OpenAI said the models used significant inference compute trying to gain wider internet access to solve the benchmark.

To do so, the models identified and exploited a zero-day flaw in the package registry cache proxy. OpenAI said it has disclosed the flaw to the vendor and is working on a patch.

After gaining broader access, the models carried out privilege escalation and lateral movement within OpenAI's research environment until they reached a machine with internet access. From there, OpenAI said, they inferred that Hugging Face might host models, datasets and solutions relevant to ExploitGym, and sought secret information that could be used to cheat the test.

In one case, OpenAI said, a model combined several attack methods, including stolen credentials and zero-day vulnerabilities, to find a remote code execution path on Hugging Face servers. OpenAI's security team first detected the unusual activity internally.

Hugging Face's security team then detected and stopped the activity on its own infrastructure. By the time the two companies connected on the response, Hugging Face had already begun containment and forensic reconstruction using its own open-source models, OpenAI said.

Controls tightened

OpenAI said it is imposing stricter infrastructure controls while patches are applied, even at the expense of research speed. It is also regularly briefing its Safety and Security Committee on those controls and their effect.

The company is continuing the forensic investigation with Hugging Face, adding stronger protections around future training and evaluation work, and bringing Hugging Face into its trusted access programme. OpenAI said it is also helping Hugging Face use its models to improve defensive work.

The incident adds to a wider debate over how developers should test increasingly capable AI systems without creating new security risks. OpenAI said the event showed that advanced models can discover and exploit new attack paths in real systems even without source-code access.

It also underscored concerns about long-horizon model behaviour, in which systems sustain multi-step operations over extended periods. OpenAI cited findings from the UK AISI that it said showed models such as GPT-5.6 Sol are increasingly able to carry out complex cyber operations over time.

The incident appears to move those concerns from theory into a live operational setting. OpenAI said the main lesson was that model security and safety measures must keep pace with the rapid rise in cyber-related model performance.

Industry implications

The case is likely to sharpen scrutiny of how AI companies conduct internal red-teaming and cyber evaluations, especially when standard refusal systems and deployment safeguards are intentionally disabled. It also raises questions about how isolated testing environments should be designed when models are pushed to pursue advanced exploitation paths.

OpenAI said the safeguards used in normal deployment were not enabled in this test because the purpose was to measure cyber vulnerabilities directly. The episode, it said, highlighted the need for stronger alignment, tighter cyber protections during evaluation and closer monitoring during internal testing.

One key unresolved issue is whether incidents of this kind will become more frequent as frontier models improve at autonomous planning and technical problem-solving. OpenAI said it expected such events to become more common as more cyber-capable models spread.

Clem Delangue, Co-founder and Chief Executive Officer of Hugging Face, commented on the joint response. "We're grateful for the collaboration with OpenAI on this and other topics. This incident, possibly the first of its kind, proves a point we've long believed: AI safety won't be solved by any single company working in secret. It will be solved in the open, collaboratively, with broad access to AI for every defender, everywhere," Delangue said.