Sam Witteveen · 2026-08-11 · notable
Sam Witteveen — 'Switchyard: NVIDIA's Local Agent Router'
Sam Witteveen's new video covers NeMo Switchyard, the Apache-2.0 model router NVIDIA released on 2026-08-11 that sends each agent step to a different backend, including local ones like vLLM and Ollama.
Sam Witteveen walks through NeMo Switchyard, NVIDIA's open router that picks a model per step of an agent run.
What is it?
NeMo Switchyard shipped on 2026-08-11, and this Sam Witteveen video is a same-day look at it. The library is an Apache-2.0 Rust proxy from NVIDIA that sits between an agent and its models, and it speaks to local backends such as vLLM and Ollama as well as hosted ones.
How does it work?
Switchyard scores each request on model capability, cost profile and infrastructure signals, then forwards it to the chosen backend. Four routers ship in the box — LLM classifier, stage, escalation and random — and a provider-agnostic SDK keeps model names separate from endpoints so backends can be swapped without touching app code.
Why does it matter?
Routing is the cheapest lever most agent builders have not pulled yet, and Sam Witteveen's audience runs exactly the local setups Switchyard targets. NVIDIA measured a 74% cost cut on a 145-task LangChain benchmark by mixing Nemotron 3.5 Lightning with Claude Opus 4.8, so a walkthrough of the setup has direct bill impact for viewers.
Who is it for?
agent developers, local-LLM builders
Try it
Watch the video, then run `uv tool install --python 3.12 "nemo-switchyard[cli,server]"`