NVIDIA NeMo

// Overview

NVIDIA NeMo Speech is an open-source framework for building speech AI models in PyTorch. It covers automatic speech recognition (ASR), text-to-speech (TTS), and speech LLMs, and ships with pre-trained checkpoints such as the Parakeet and Canary model families that you can download from HuggingFace and run directly.

It is aimed at researchers and PyTorch developers who want to create, customize, or deploy speech models without starting from scratch. You can use the released checkpoints for inference out of the box, or fine-tune and train your own models using the framework's building blocks.

Within the speech and audio space, NeMo focuses on the model side rather than being a lightweight inference-only library. It works on top of your existing Python, PyTorch, and CUDA stack, and an NVIDIA GPU is recommended for inference and required for training.

// What it does

• Pre-trained ASR checkpoints including the Parakeet and Canary families, with multilingual recognition and translation support
• Text-to-speech models such as MagpieTTS, with multilingual voice synthesis
• Streaming ASR options with controllable latency for real-time transcription
• Simple model loading from HuggingFace via from_pretrained and a one-call transcribe API
• Works on top of your own Python, PyTorch, and CUDA versions instead of replacing them
• Open source under Apache 2.0, built for both inference and custom training

// Getting started

Install NeMo Speech, then load a pre-trained model and transcribe an audio file. A recent NVIDIA GPU with CUDA is recommended for inference.

git clone https://github.com/NVIDIA-NeMo/NeMo.git
cd NeMo
uv sync --extra all --extra cu13     # CUDA 13.x

pip install 'nemo-toolkit[asr,tts,cu13]' --extra-index-url https://download.pytorch.org/whl/cu132

import nemo.collections.asr as nemo_asr

asr_model = nemo_asr.models.ASRModel.from_pretrained(
    model_name="nvidia/parakeet-tdt-0.6b-v3"
)

output = asr_model.transcribe(['2086-149220-0033.wav'])
print(output[0].text)

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Transcribe audio or video into text using a pre-trained Parakeet or Canary model
• Add real-time, low-latency streaming speech recognition to an application
• Generate multilingual speech from text with a TTS model like MagpieTTS
• Fine-tune or train custom ASR/TTS models on your own data

// How NVIDIA NeMo compares

NVIDIA NeMo alongside other open-source audio, music & voice tools AI/TLDR tracks, ranked by GitHub stars.