NVIDIA · 2026-06-04 · major
NVIDIA Nemotron 3.5 ASR Streaming 0.6B — Open-Weights Multilingual Speech Model Sustains ~17× More Concurrent Streams on H100, Covers 40 Locales From One Checkpoint
NVIDIA released a 600M-parameter streaming ASR model under OpenMDW-1.1. Cache-aware FastConformer-RNNT processes each audio frame exactly once at 80ms–1.12s latency, transcribes 40 locales with built-in punctuation, and beats prior streaming models by ~17× on H100 throughput.
A 600M-param open ASR model that streams 40 locales from one checkpoint and squeezes ~17× more concurrent voice sessions onto an H100.
Key specs
| License | OpenMDW-1.1 |
|---|---|
| Parameters | 600M |
| Language locales | 40 |
| Concurrent stream gain | ~17× |
| Min latency | 80ms |
| Max latency | 1.12s |
What is it?
Nemotron 3.5 ASR is NVIDIA's first open multilingual streaming speech recognizer, successor to the English-only Nemotron 3 ASR. One checkpoint handles 40 language-locales including English, Spanish, German, French, Japanese, Korean and Mandarin, with built-in punctuation and casing.
How does it work?
The model uses a Cache-Aware FastConformer-RNNT encoder that processes each audio frame exactly once and reuses cached encoder states for new chunks, eliminating the overlapping computation found in traditional buffered streaming. Runtime-tunable att_context_size lets developers pick latency points between 80ms and 1.12s. Language can be passed as an explicit tag or auto-detected.
Why does it matter?
Real-time voice agents have been bottlenecked on either accuracy (cloud APIs) or throughput (open streaming models). A 0.6B model that fits 17× more concurrent streams onto an H100 collapses the per-session cost of voice assistants, call-center transcription and live captioning. OpenMDW-1.1 weights are commercial-friendly out of the box.
Who is it for?
voice-agent and call-center teams, real-time captioning products, on-prem deployers
Try it
pip install nemo_toolkit['asr']>=26.06 && load nvidia/nemotron-3.5-asr-streaming-0.6b