MOSS-TTS

// Overview

MOSS-TTS is an open-source speech and sound generation model family from MOSI.AI and the OpenMOSS team. Rather than one checkpoint it is a set of models aimed at different jobs: MOSS-TTS-v1.5 and MOSS-TTS-Local-Transformer-v1.5 for multilingual long-form narration and voice cloning, MOSS-TTSD for multi-speaker dialogue, podcasts and dubbing, MOSS-TTS-Realtime for streaming speech, MOSS-VoiceGenerator for voice design, MOSS-SoundEffect for environmental audio, and MOSS-TTS-Nano — roughly 100M parameters — for CPU and in-browser synthesis.

The generation API is deliberately small: a `generate` call covering direct synthesis from Chinese, English, mixed text with language tags, Pinyin or IPA; voice cloning from a reference clip; duration control; and explicit pause insertion with a `[pause X.Ys]` marker. v1.5 added stronger multilingual synthesis when a language tag is supplied, more stable cloning, better long-reference short-text cloning and punctuation-driven prosody. The Local-Transformer v1.5 checkpoint scales the backbone from Qwen3-1.7B to Qwen3-4B and pairs it with MOSS-Audio-Tokenizer-v2 for native 48 kHz stereo output.

Serving is where the family has invested most. vLLM-Omni supports the full series across the MossTTSDelay, MossTTSRealtime and MossTTSNano architectures; SGLang added a MossTTSDelay backend for roughly 3× faster generation, and SGLang-Omni gave MOSS-TTS-Local-Transformer-v1.5 day-zero support with an OpenAI-compatible /v1/audio/speech endpoint, streaming and voice cloning. There is also a first-class llama.cpp implementation for GGUF backbone inference with ONNX codec decoding, plus mlx-audio support on Apple silicon. Weights are on Hugging Face and ModelScope, and fine-tuning tutorials cover both the MossTTSLocal architecture and the realtime model.

// What it does

• One family covering long-form narration, multi-speaker dialogue, voice design, sound effects and real-time streaming
• Voice cloning from a reference clip, with stable identity on long references and short target text
• Multilingual synthesis with explicit language tags, plus Pinyin and IPA input
• Duration control and explicit pause insertion via a [pause X.Ys] marker
• MOSS-Audio-Tokenizer-v2 for native 48 kHz stereo input and output
• MOSS-TTS-Nano at ~100M parameters runs multilingual cloning and streaming on four CPU cores or in a browser
• Served by vLLM-Omni, SGLang and SGLang-Omni (OpenAI-compatible /v1/audio/speech), llama.cpp GGUF, and mlx-audio
• Gradio demo scripts per model plus fine-tuning tutorials for the MossTTSLocal and realtime architectures

// Getting started

The repository is installed from source. The team recommends a clean Python 3.12 environment with Transformers 5.0.0 and ffmpeg, which torchcodec needs for audio I/O. Dependencies in pyproject.toml pin torch and torchaudio 2.9.1+cu128.

# Debian/Ubuntu
sudo apt-get install -y ffmpeg
# macOS
brew install ffmpeg

conda create -n moss-tts python=3.12 -y
conda activate moss-tts
git clone https://github.com/OpenMOSS/MOSS-TTS.git
cd MOSS-TTS
pip install --extra-index-url https://download.pytorch.org/whl/cu128 -e ".[torch-runtime]"

git clone https://github.com/OpenMOSS/MOSS-TTS.git
cd MOSS-TTS
uv venv --python 3.12 .venv
source .venv/bin/activate
uv pip install --torch-backend cu128 -e ".[torch-runtime]"

pip install --extra-index-url https://download.pytorch.org/whl/cu128 -e ".[torch-runtime,flash-attn]"

import torch
from transformers import AutoModel, AutoProcessor

# Disable the broken cuDNN SDPA backend, keep the fallbacks enabled
torch.backends.cuda.enable_cudnn_sdp(False)
torch.backends.cuda.enable_flash_sdp(True)
torch.backends.cuda.enable_mem_efficient_sdp(True)

pretrained_model_name_or_path = "OpenMOSS-Team/MOSS-TTS-v1.5"
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if device == "cuda" else torch.float32

processor = AutoProcessor.from_pretrained(pretrained_model_name_or_path)

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Reach for it to narrate long-form multilingual text where cloned voice identity has to stay stable across chapters
• Reach for it to synthesise multi-speaker dialogue for podcasts or dubbing rather than a single narrator track
• Reach for it when speech has to stream in real time behind an OpenAI-compatible audio endpoint
• Reach for it when TTS must run on CPU or in a browser — MOSS-TTS-Nano is built for that footprint

// How MOSS-TTS compares

MOSS-TTS alongside other open-source audio, music & voice tools AI/TLDR tracks, ranked by GitHub stars.