Is NeMo Switchyard ready for production use?

No. The NeMo Switchyard repository labels the project pre-alpha experimental software and says it is not recommended for production. NVIDIA is shipping it open source under Apache-2.0 so teams can test routing on their own traffic, but the API surface and the four bundled routers should be treated as unstable for now.

How much money does NeMo Switchyard actually save?

NVIDIA reports two measured results for NeMo Switchyard. On a LangChain set of 145 multi-turn tasks, routing between Nemotron 3.5 Lightning and Claude Opus 4.8 cut cost 74% for roughly a 6-point accuracy tradeoff. On Cognition's FrontierCode, routing between Opus 5 and Kimi K2.7 hit 50.6% accuracy at $3.11 mean cost, about 28% below Opus 5 alone.

Which agent tools and partners support NeMo Switchyard?

NeMo Switchyard plugs into Claude Code, Codex CLI, OpenClaw, vLLM, NVIDIA NIM, Ollama and any OpenAI-compatible endpoint. NVIDIA lists Boomi, Cadence, Classmethod, Cognition, Kong, LangChain, LiteLLM, Nous Research, Ramp and Siemens as partners that have integrated it or are rolling it out.

Can I write my own routing rules for NeMo Switchyard?

Yes. NVIDIA says developers can bring their own routing algorithms or customization data to NeMo Switchyard rather than being limited to the four bundled routers. The library also translates between the OpenAI and Anthropic API shapes and exports Prometheus metrics for requests, errors, latency and tokens.

NVIDIA · 2026-08-11 · major

NeMo Switchyard — NVIDIA's open router picks a model per agent step

NeMo Switchyard is an Apache-2.0 routing library NVIDIA released on 2026-08-11. It sends each step of an agent run to whichever model fits it, using capability, cost and infrastructure signals, without rewriting the app.

NVIDIA NeMo Switchyard agentic AI model routing announcement graphic

An Apache-2.0 Rust proxy from NVIDIA that reshuffles which model handles each step of an agent run.

Key specs

GitHub stars	241
Cost cut (lang chain test)	74%

Quick facts

Maker	NVIDIA
License	Apache-2.0
Built in	Rust, with Python CLI bindings
Routers included	LLM Classifier, Stage, Escalation, Random
Works with	Claude Code, Codex CLI, OpenClaw, vLLM, NVIDIA NIM, Ollama
Install	uv tool install --python 3.12 "nemo-switchyard[cli,server]"
Status	Pre-alpha, not recommended for production

What is it?

Model routing gets a first-party open-source option: NeMo Switchyard, released by NVIDIA on 2026-08-11, sits between an agent and its models and picks a different backend per request. It ships four routers out of the box — an LLM classifier, a stage router, an escalation router that judges weak-to-strong, and a random splitter for A/B tests.

How does it work?

Each request is scored on three signal groups: which models can solve the task, what each one costs in latency and money, and system-level reliability. A provider-agnostic SDK keeps semantic model names separate from provider endpoints, so swapping a backend does not touch application code. The proxy also translates between OpenAI and Anthropic API formats and exports Prometheus metrics.

Why does it matter?

Most agent runs mix trivial steps with a few hard ones, but teams usually pay frontier rates for all of them. NVIDIA's own tests put the saving at 74% on a 145-task LangChain set, and Cognition cut mean cost 28% in Devin Desktop. Because it speaks to Claude Code, Codex CLI, OpenClaw, vLLM and Ollama, the change is a proxy swap rather than a rewrite.

Who is it for?

agent developers, platform teams watching inference bills

Frequently asked questions

Is NeMo Switchyard ready for production use?: No. The NeMo Switchyard repository labels the project pre-alpha experimental software and says it is not recommended for production. NVIDIA is shipping it open source under Apache-2.0 so teams can test routing on their own traffic, but the API surface and the four bundled routers should be treated as unstable for now.
How much money does NeMo Switchyard actually save?: NVIDIA reports two measured results for NeMo Switchyard. On a LangChain set of 145 multi-turn tasks, routing between Nemotron 3.5 Lightning and Claude Opus 4.8 cut cost 74% for roughly a 6-point accuracy tradeoff. On Cognition's FrontierCode, routing between Opus 5 and Kimi K2.7 hit 50.6% accuracy at $3.11 mean cost, about 28% below Opus 5 alone.
Which agent tools and partners support NeMo Switchyard?: NeMo Switchyard plugs into Claude Code, Codex CLI, OpenClaw, vLLM, NVIDIA NIM, Ollama and any OpenAI-compatible endpoint. NVIDIA lists Boomi, Cadence, Classmethod, Cognition, Kong, LangChain, LiteLLM, Nous Research, Ramp and Siemens as partners that have integrated it or are rolling it out.
Can I write my own routing rules for NeMo Switchyard?: Yes. NVIDIA says developers can bring their own routing algorithms or customization data to NeMo Switchyard rather than being limited to the four bundled routers. The library also translates between the OpenAI and Anthropic API shapes and exports Prometheus metrics for requests, errors, latency and tokens.

Try it

uv tool install --python 3.12 "nemo-switchyard[cli,server]"