AuK

// Overview

AuK is an open-source speech foundation model from Tencent Hunyuan, released on 9 September 2026 with its technical report, source code and weights. It is 1.5B parameters, and the point of it is a single interface: instead of one model for text-to-speech, another for denoising and another for source separation, AuK takes a natural-language instruction plus audio context and covers all of them.

Five task families are supported, and the training set was built to match: roughly 3.03 billion instruction–audio instances and 1.95 million hours of effective supervision across speech generation, content editing, enhancement and separation, paralinguistic editing, and acoustic editing. In practice that means zero-shot and instruction-controlled text-to-speech; replacing, inserting or removing words in an existing recording (including lyrics in singing); changing pitch, speed and volume; changing emotion, timbre, accent, nonverbal sounds or converting to a whisper; and denoising, speaker extraction and music separation.

Architecturally, a multimodal LLM supplies semantic conditioning and a VAE trained jointly on speech, general audio and music supplies acoustic conditioning, both feeding a hybrid rectified-flow Transformer built from dual-stream MMDiT blocks followed by unified single-stream DiT blocks. Training runs generation-only warm-up, then joint generation–editing pre-training, then two post-training passes: human-feedback preference optimization for open-ended editing and reward-based reinforcement learning for generation. A distilled sibling, AuK-Flash, runs 4-step inference without classifier-free guidance and is reported at a 4.5× wall-clock speedup over the full model under matched conditions. Both are MIT-licensed and published on Hugging Face and ModelScope.

// Strengths

• One natural-language instruction interface across generation, editing, enhancement and separation, instead of a separate model per task
• MIT licence on both code and weights — among the most permissive terms for a speech model of this capability
• Small enough to self-host at 1.5B parameters
• AuK-Flash gives a 4.5× wall-clock speedup via 4-step inference without classifier-free guidance
• Paralinguistic control that goes beyond voice cloning: emotion, timbre, de-accenting, nonverbal sounds and whisper conversion
• Trained on roughly 3.03 billion instruction–audio instances and 1.95 million hours of effective supervision

// Best for

• Instruction-driven text-to-speech where the prompt, not a parameter grid, sets delivery and emotion
• Fixing a recording in place — replacing a misspoken word or a lyric without re-recording the take
• Cleaning up audio: denoising, separating speech from music, or extracting one speaker from a mix
• Self-hosted speech research and products that need permissive licensing on both code and weights

// How to access