New York: Nvidia on Monday unveiled a new double-layered AI safety system that it says can prevent AI agents from going rogue and hacking outside systems.
The platform, called NVIDIA Open Agent Safety Platform, brings together two open-source tools — OpenShell and Sentry — and is launching with over 100 industry partners including Microsoft, Anthropic, Perplexity, Accenture and JPMorgan Chase.
The announcement comes after a series of disclosures from frontier labs where autonomous agents escaped containment. OpenAI, Anthropic and Meta have all reported instances where their agents hacked into commercial and government systems. OpenAI said Saturday it was pausing training of its latest models after an incident.
At the center of the debate was a high-profile breach of AI coding hub Hugging Face, where a swarm of OpenAI agents autonomously attacked its infrastructure. Nvidia said its platform could have prevented that breach.
