Red Hat and NVIDIA on Tuesday at GTC Berlin unveiled a layered security architecture for autonomous AI agents built around a principle that has governed operating-system security for three decades — that no process should be trusted to enforce its own constraints — applying that foundation for the first time to enterprise AI deployments. The joint announcement, covering an open-source agent sandbox runtime, a hardware-level memory forensics system running on NVIDIA’s BlueField data processing units, and a model governance reference design, arrives as a Deloitte survey of 3,235 business and technology leaders across 24 countries finds only 21% of organizations have a mature agentic AI governance model — and as Gartner has named agentic AI oversight its top cybersecurity trend for 2026.

Agents That Can’t Be Trusted to Police Themselves

Autonomous AI agents — systems that plan, invoke tools, and execute multi-step tasks without constant human oversight — are increasingly deployed with broad permissions over enterprise data and APIs. That scope creates two distinct failure modes: agents that exceed their authorized boundaries through misconfiguration, and agents that are actively compromised by prompt injection attacks, poisoned model weights, or supply-chain exploits targeting the software they depend on.

Prompt injection remains OWASP’s top LLM security risk, according to the organization’s LLM Top 10. A Cloud Security Alliance research note published in April 2026 documented a 32% rise in malicious IPI on crawled web pages between November 2025 and February 2026, based on Google data. Palo Alto Networks Unit 42 catalogued 22 distinct payload-delivery techniques across 12 confirmed indirect injection cases against AI agents. The UK’s National Cyber Security Centre has stated that prompt injection may represent an inherent LLM design flaw with no surefire mitigations — a conclusion that points directly toward architectural remediation rather than software filtering.

The traditional response to agent misbehavior — embedding guardrails inside the agent’s own code — is, by Red Hat and NVIDIA’s framing, structurally insufficient. “An agent shouldn’t enforce its own boundaries,” John Fanelli, NVIDIA’s vice president of enterprise software, said in a statement accompanying the announcement. “Those controls belong outside the agent, in the runtime and the infrastructure beneath it.” That philosophy shapes every component of Tuesday’s announcement.

OpenShell: Enforcement Controls That Live Outside the Agent’s Reach

The first layer of the architecture is NVIDIA OpenShell, an open-source (Apache 2.0) agent sandbox runtime built in Rust, whose reference architecture Red Hat is working to integrate natively into its AI platforms as a first-class capability.

OpenShell uses a combination of three Linux kernel mechanisms to enforce its boundaries: Linux Landlock, a kernel-enforced filesystem isolation module; seccomp BPF, which filters the system calls a process is permitted to make; and network namespaces that isolate the agent’s network context. Together these mechanisms create an enforcement boundary at the OS kernel level — outside the agent’s code — that the sandboxed process cannot modify, disable, or escape from within.

This is not a new architectural principle. It is the same principle that governs browser sandboxing, Android per-app isolation, and every other robust computing sandbox: enforcement in the kernel, not in the application. What OpenShell adds is a policy engine that runs as a deliberately out-of-process component — meaning it executes outside the agent itself. A compromised or misbehaving agent cannot reach the enforcement mechanism. Filesystem and process policies are locked at sandbox creation time; network and inference policies can be updated on a running sandbox without restart.

API credentials provide a concrete example of why this matters. Raw API keys for model providers are held on the host — outside the sandbox — and routed to those providers through the OpenShell gateway. The sandboxed agent receives model responses but never handles the underlying credentials. Even if the agent’s code is compromised via prompt injection and attempts to exfiltrate API keys, those keys were never inside its memory space to take. NemoClaw, NVIDIA’s blueprint lifecycle manager for OpenShell, handles sandbox policy lifecycle management across the Landlock, seccomp BPF, and network namespace enforcement layers.

How Does Hardware-Level Isolation Make AI Agent Monitoring Tamper-Proof?

The second layer of the architecture operates beneath the operating system. NVIDIA Sentry is a security framework that runs not on the host server’s main CPU, but on the Arm processors inside NVIDIA’s BlueField data processing units — purpose-built network and security accelerator cards that sit between a server and the network fabric and run their own OS, entirely separate from the host.

Sentry deploys NVIDIA DOCA Argus (current version: 1.5.0-doca3.5.0), software that performs real-time host memory forensics via DPU, inspecting selected host memory structures directly from the DPU’s isolated hardware domain. The architecture is agentless: Argus observes processes, files, and network activity on the host without requiring any monitoring software to be installed or trusted on the host itself. Because the monitoring runs on a processor that is physically separate from the host CPU — with its own memory space and its own OS — a compromised workload on the host has no mechanism to reach, disable, or spoof the monitoring system. DOCA Argus integrates with enterprise SIEM, SOAR, and XDR platforms for alert routing.

When Argus detects anomalous behavior — an agent spawning unexpected processes, initiating unauthorized network connections, or exhibiting behavioral indicators of compromise — it can trigger BlueField network policies that isolate the affected workload. Red Hat says the system can quarantine a compromised agent in milliseconds, a response window that is critical: indirect prompt injection attacks have been observed cascading through multi-agent pipelines within seconds of initial compromise.

The architecture builds on BlueField-3 support already present in Red Hat OpenShift 4.22. A secondary benefit: offloading monitoring and network security functions to the DPU frees host CPU capacity that would otherwise be consumed by security agents, leaving more headroom for AI inference workloads themselves.

Model Governance: Decoupling Access Policy From Inference Infrastructure

The third pillar of the announcement addresses model access control at the software layer. A joint reference design combines Red Hat’s AI Gateway with NVIDIA’s Inference Hub reference design, integrating across NVIDIA’s inference software stack: vLLM, llm-d, NVIDIA Dynamo, and NVIDIA NIM microservices. This reference design was detailed in Tuesday’s announcement.

The governance layer provides self-service model discovery, access policies, rate limiting, and subscription entitlement tracking, while keeping access governance separate from model execution. The architectural separation is deliberate: by decoupling who can call a model from the infrastructure that runs it, enterprises can adjust compute quotas, attribute costs to specific teams or applications, and maintain performance guarantees across hybrid cloud environments without modifying the underlying inference stack each time a policy changes. This separation is directly relevant to emerging compliance requirements under the EU AI Act, which sets audit and oversight rules for high-risk AI systems.

The companies also announced joint support for NVIDIA Nemotron 3.5 Super VL — a multimodal model — in the vLLM inference engine and in Red Hat AI Factory with NVIDIA, using quantization and speculative decoding to reduce inference latency while preserving output quality.

Vera Rubin Readiness and the Partner Delivery Stack

Tuesday’s announcement also extended the partnership’s hardware roadmap. RHEL for NVIDIA 26.03, released in March 2026, adds support for NVIDIA’s Vera CPUs and is positioned as the operating system layer for NVIDIA’s upcoming Vera Rubin data center platform. Red Hat described the release as modifying a minimal set of core packages to accommodate new hardware while preserving the standard RHEL userspace — including SELinux — and giving organizations a path to transition to mainline RHEL once the Vera hardware patches mature and enter the upstream Linux kernel. That commitment to avoiding a permanent fork is significant: enterprises evaluating Vera Rubin deployments can adopt RHEL for NVIDIA 26.03 without trading long-term supportability for early hardware access.

The underlying platform architecture separates AI workloads running on host CPUs from networking and security services running on BlueField’s Arm processors. When configured to restrict host access to the DPU, that hardware boundary protects infrastructure controls even if an attacker gains a foothold on the host — a property that becomes more valuable as AI agents are granted broader permissions.

Hardware partners Cisco Systems, Dell Technologies, and Lenovo will deliver the joint use cases and reference designs to enterprise customers. Red Hat’s native OpenShell integration into its AI platforms remains in progress.

What GTC Berlin’s Scale Signals About the Industry’s Urgency

The timing of Tuesday’s announcement — at NVIDIA’s first major European GTC event, which drew more than 10,000 attendees to STATION-Berlin and featured a Jensen Huang keynote at Berlin’s Tempodrom — reflects the global scale at which both companies are operating. GTC Berlin’s programming centered on agentic AI, robotics, and AI infrastructure, the domains where the security questions raised by Tuesday’s announcement are most acute.

For enterprise security teams, the announcement frames a procurement question that will become unavoidable as agentic AI moves from pilot to production: is the vendor’s security model based on software guardrails inside the agent, or on enforcement controls the agent cannot reach? The Deloitte survey data makes the governance gap concrete — 74% of organizations expect moderate to extensive AI agent use by 2027, while only 21% currently have mature governance frameworks in place. Tuesday’s announcement from Red Hat and NVIDIA is one answer to the question of how to close that gap before the deployments arrive.

Frequently Asked Questions

What makes OpenShell’s security model different from software guardrails built into an AI agent?

Software guardrails embedded in agent code are enforced by the agent itself — meaning a compromised agent can disable, modify, or circumvent them. OpenShell’s policy engine runs as a deliberately out-of-process component, enforced by the Linux kernel through Landlock LSM and seccomp BPF mechanisms. The sandboxed agent has no access to the enforcement layer. A compromised agent cannot change the rules it runs under, because those rules are implemented and enforced outside its execution environment — the same architectural principle that underlies browser sandboxing and OS-level process isolation.

How does DOCA Argus detect a compromised AI agent without installing monitoring software on the host?

DOCA Argus runs on the Arm processors inside NVIDIA’s BlueField DPU — a separate hardware card that operates with its own OS, its own memory space, and its own network ingress/egress control, physically separate from the host CPU. The DPU can read selected host memory structures via hardware-controlled direct memory access without any monitoring agent installed on or trusted by the host. Because the monitoring system exists in a separate hardware domain, a compromised host workload cannot reach or disable it. When Argus detects anomalous behavior, BlueField’s network policies can quarantine the affected workload in milliseconds, before a prompt injection attack cascades through a multi-agent pipeline.

What is the biggest AI security risk for enterprises deploying autonomous agents in 2026?

Gartner named agentic AI oversight its top cybersecurity trend for 2026, specifically flagging the risk of unsanctioned agents operating without defined decision boundaries or real-time behavioral monitoring. On the attack side, prompt injection — particularly indirect injection, where malicious instructions are embedded in content an agent retrieves from the web or external systems — has emerged as the leading exploitation technique. The Cloud Security Alliance documented a 32% rise in malicious IPI on crawled web pages in the six months ending February 2026. The governance gap compounds the risk: a Deloitte survey found 74% of organizations plan significant AI agent deployments by 2027, while only 21% have mature governance frameworks today.

Will the OpenShell integration work with existing Red Hat OpenShift deployments?

Red Hat is working to integrate OpenShell natively into its AI platforms — including OpenShift and Red Hat Enterprise Linux — as a standard deployment option rather than an add-on. The integration is in progress as of Tuesday’s announcement; it is not yet generally available as a native Red Hat platform feature. The underlying DPU security layer (Sentry/DOCA Argus) builds on BlueField-3 support in OpenShift 4.22. Organizations that want to evaluate the full architecture today can do so through the reference designs being delivered by hardware partners Cisco, Dell, and Lenovo.