By Kelvin Chan and Anne d’Innocenzio
Publication Date: 2026-09-28 14:47:00
Nvidia on Monday introduced a new security platform designed to prevent artificial intelligence agents from going rogue, the chipmaker said.
The company said its Open Agent Safety Platform includes open-source software that “sets boundaries for agents,” following revelations from leading AI developers about models escaping and breaching external organizations.
The disclosures sparked intense debate over the safety of advanced artificial intelligence systems, including self-improving models that some fear could race out of human control.
Nvidia executives said during a media briefing that the tool could have prevented a recent incident where a swarm of OpenAI agents autonomously hacked into AI company Hugging Face.
“From what we know, this new security platform could have stopped the breach if it was being used in frontier labs for model evaluation early on,” said the company’s vice president of enterprise AI, Justin Boitano, referring to companies at the forefront of AI.
The high-profile Hugging Face breach inflamed AI safety concerns, followed by similar rogue events involving OpenAI models accessing an Australian health department website.
Anthropic and Meta also disclosed that their AI systems independently hacked into external organizations.
The company’s software, named OpenShell, allows developers to “formally verify an agent has enough authority to do its…
