Red Hat · 2026-05-12 · major
Red Hat AI 3.4 — Model-as-a-Service, AgentOps, and vLLM Distributed Inference With 2–3× Speculative Decoding; Coming to AWS, Azure, GCP, and IBM Cloud
At Red Hat Summit 2026 in Atlanta, Red Hat unveiled AI 3.4 — a metal-to-agent enterprise platform with Model-as-a-Service, AgentOps tooling, vLLM-powered distributed inference, and SPIFFE/SPIRE agent identity.
Red Hat's annual platform release moves from experimental chatbots to a hardened, governed runway for autonomous agents in production.
Key specs
| Speculative decoding speedup | 2–3x |
|---|---|
| Summit location | Atlanta |
| Marketplaces | AWS, Azure, GCP, IBM Cloud |
What is it?
Red Hat AI 3.4 is the latest version of Red Hat's enterprise AI platform, focused on operating AI agents at scale rather than training new models. It bundles a Model-as-a-Service layer, agent lifecycle tooling, distributed inference, and built-in security and governance into one suite that runs on OpenShift across hybrid cloud environments.
How does it work?
Inference runs on vLLM with the llm-d distributed engine, which adds request prioritization, autoscaling against GPU utilization, and speculative decoding that Red Hat claims gives 2–3× faster responses. AgentOps adds tracing, MLflow-based prompt management, an evaluation hub, and an MCP gateway for tool access. Agents authenticate with short-lived SPIFFE/SPIRE identities, and adversarial scanning runs through Garak plus NVIDIA NeMo Guardrails. Day-zero support targets NVIDIA Blackwell and AMD MI355X.
Why does it matter?
Enterprise teams trying to deploy agents have so far stitched together vLLM, observability, identity, and red-teaming themselves. AI 3.4 packages those plumbing concerns into one supported product on OpenShift, with Red Hat AI Inference launching on IBM Cloud and Red Hat AI Enterprise heading to AWS, Azure, and Google Cloud marketplaces.
Who is it for?
Platform and ML teams running agentic workloads on OpenShift or hybrid cloud.
Try it
https://www.redhat.com/en/technologies/ai/enterprise