# Red Hat AI 3.4 — Model-as-a-Service, AgentOps, and vLLM Distributed Inference With 2–3× Speculative Decoding; Coming to AWS, Azure, GCP, and IBM Cloud

> Red Hat's annual platform release moves from experimental chatbots to a hardened, governed runway for autonomous agents in production.

- Org: Red Hat
- Date: 2026-05-12
- Category: tool, ecosystem
- Source: https://www.redhat.com/en/blog/inference-agentic-ai-scaling-enterprise-foundation-red-hat-ai-34
- Canonical page: https://ai-tldr.dev/releases/redhat-ai-34-summit-2026/

## In short

At Red Hat Summit 2026 in Atlanta, Red Hat unveiled AI 3.4 — a metal-to-agent enterprise platform with Model-as-a-Service, AgentOps tooling, vLLM-powered distributed inference, and SPIFFE/SPIRE agent identity.

## What it is

Red Hat AI 3.4 is the latest version of Red Hat's enterprise AI platform, focused on operating AI agents at scale rather than training new models. It bundles a Model-as-a-Service layer, agent lifecycle tooling, distributed inference, and built-in security and governance into one suite that runs on OpenShift across hybrid cloud environments.

## How it works

Inference runs on vLLM with the llm-d distributed engine, which adds request prioritization, autoscaling against GPU utilization, and speculative decoding that Red Hat claims gives 2–3× faster responses. AgentOps adds tracing, MLflow-based prompt management, an evaluation hub, and an MCP gateway for tool access. Agents authenticate with short-lived SPIFFE/SPIRE identities, and adversarial scanning runs through Garak plus NVIDIA NeMo Guardrails. Day-zero support targets NVIDIA Blackwell and AMD MI355X.

## Why it matters

Enterprise teams trying to deploy agents have so far stitched together vLLM, observability, identity, and red-teaming themselves. AI 3.4 packages those plumbing concerns into one supported product on OpenShift, with Red Hat AI Inference launching on IBM Cloud and Red Hat AI Enterprise heading to AWS, Azure, and Google Cloud marketplaces.

## Who it's for

Platform and ML teams running agentic workloads on OpenShift or hybrid cloud.

## Try it

https://www.redhat.com/en/technologies/ai/enterprise

## Links

- [Red Hat Announcement](https://www.redhat.com/en/blog/inference-agentic-ai-scaling-enterprise-foundation-red-hat-ai-34)
- [Press Release](https://www.redhat.com/en/about/press-releases/red-hat-unites-builders-and-operators-agentic-future-major-advancements-red-hat-ai)
- [InfoWorld Coverage](https://www.infoworld.com/article/4169801/red-hats-message-to-enterprises-you-dont-need-to-re-platform-for-ai-agents.html)
- [AIwire Coverage](https://www.hpcwire.com/aiwire/2026/05/12/red-hat-learns-new-ai-tricks-at-summit-2026/)