NVIDIA Unveils an Open Platform for AI Agent Safety
NVIDIA has partnered with over 100 industry companies, including Anthropic and SpaceXAI, to introduce a security layer that operates across both software and hardware.
NVIDIA, together with over 100 industry partners, launches a system to harness AI agents, an Open Agent Safety Platform, bringing together OpenShell and Sentry and described as a safety enforcement layer that works across software and hardware.
In an announcement, NVIDIA explains that AI agents can sometimes run out of control, reach systems they never should have been allowed to, and misreport what they did. NVIDIA sees a need to accelerate AI safety research and engineering.
NVIDIA CEO Jensen Huang emphasized the point and wrote on X that though AI is an "extraordinary" technology that enables advanced research, its full potential can only be realized when it's built to be safe:
"Artificial intelligence is extraordinary technology that will advance discovery, productivity, security, health, and prosperity for generations to come."
"But its full promise can only be realized when people have confidence that AI is being built to be safe and deployed with wisdom and responsibility."
According to the CEO, NVIDIA's new product is the beginning of an open ecosystem intended "to build the trust layer for safe agent systems."
The CEO wrote in a follow-up post that the Open Agent Safety Platform Reference Design combines NVIDIA OpenShell and NVIDIA Sentry. OpenShell is an open-source secure runtime that controls AI agent performance by giving them boundaries, tracing their actions, and enforcing policy as they work. NVIDIA Sentry is a hardware-based security system that monitors AI agents as they run; it can detect when an agent tries to go beyond its allowed software boundaries and quickly isolate or stop it.
Partners include Anthropic (Claude), SpaceXAI (Cursor and Grok), Scale AI (Scale GenAI Portfolio), and SAP (Joule Studio), which are integrating NVIDIA's agent safety technologies into their AI products and infrastructure. Other companies working with NVIDIA include Microsoft, IBM, Hugging Face, Perplexity, ServiceNow, Cisco, CrowdStrike, Palantir, and Palo Alto Networks, among more than 100 organizations.
Earlier, NVIDIA CEO Jensen Huang shared an open letter titled "Open Weights and American AI Leadership," which discusses why open-weight AI models, which can be downloaded, inspected, modified, and run on their own machine, matter. Major tech companies signed it, including Google, AMD, Cisco, Cloudflare, GitHub, Microsoft, Meta, IBM, Dell Technologies, Palantir, Hugging Face, Mistral, Andreessen Horowitz, Y Combinator, Perplexity, ServiceNow, CrowdStrike, and the Linux Foundation.