In what OpenAI creator ChatGPT is calling an “unprecedented cyber incident,” an artificial intelligence software “went rogue” and escaped, gained access to the internet and hacked into a start-up company.
Cybersecurity expert Richard Ford, chief technology officer at Integrity360, told the Daily Mail: “This is the moment many in cybersecurity have been warning about.”
The publication reported an “autonomous agent” was being tested but found vulnerabilities and “managed to escape containment before reaching the internet and breaking into Hugging Face,” a hub for sharing AI models.
It then compromised the hub’s infrastructure.
“Until now, we’ve seen attackers use AI to automate parts of an attack, but this is one of the first public examples of an AI agent independently identifying a weakness, escaping what should have been a secure environment and attempting to compromise another organization,” Ford said. “It also reinforces that AI doesn’t replace the fundamentals of cyber security. The agent exploited a vulnerability in what should have been a secure sandbox, showing that good cyber hygiene, robust access controls and effective guardrails remain essential.”
The report said OpenAI had been testing the software “by setting tasks in a controlled digital testing ground, where internet access was limited.”
However, the code created “an unprecedented cyber incident” in the scenario.
“The company said in a blog post last week that it used Zhipu AI’s GLM-5.2 for the analysis, which also allowed it to keep attacker data and any credentials within its systems,” the report said.
Hugging Face co-founder Thomas Wolf told the publication, “When a frontier model is attacking you and moving laterally inside your infrastructure, defenders need wide access to near-frontier tools within hours or even minutes, rather than being pointed towards a closed-door, vetted application program for model access.”
OpenAI chief executive Sam Altman said, “We had a significant security incident during evaluation of our models.”
Clément Delangue, of Hugging Face, said, “It’s quite mind-blowing that all of this happened autonomously.”
OpenAI reported the software used stolen credentials and found a previously unknown vulnerability to access Hugging Face servers.
Katie Moussouris, chief executive of Luta Security, warned more breaches are coming.
Hellbound and Down 