YuE2

// Overview

YuE2 is an open music generation system from Multimodal Art Projection (m-a-p). It takes lyrics and a style prompt and produces a complete song with vocals and accompaniment. What separates it from most text-to-music systems is the middle step: YuE2 first writes a symbolic score you can read, play and change, and only renders audio once that plan is settled. The project calls this white-box composition.

The system is built around YuE2-3B, a 3B-parameter AR-NAR Mixture-of-Transformers model that generates the symbolic score and semantic tokens. Flow matching then produces acoustic latents, which a VAE decodes to 48 kHz stereo audio. Several companion models ship alongside it: YuE2-Vae as the default decoder, SheetSage2 for transcribing existing audio back into a score, and MERT-v2-FullSong for music representations. On the team's own WildSongBench set of 192 prompts, YuE2 with best-of-8 selection reports a 6.9632 SongBench average, above Suno v5 (6.8721) and Suno v6 (6.5562).

Licensing is split. The first-party code, agent skill and documentation are Apache-2.0, so the software can be reused freely. The model weights are separately licensed under CC BY-NC 4.0, which rules out commercial use. The team states its models are trained primarily on CC0 music and synthetic data, with most of the synthetic data licensed from Tokenwave.AI.

// What it does

• Full-song generation from lyrics plus a style prompt, with vocals and accompaniment at 48 kHz stereo
• Editable symbolic composition — read, play and change the melody and chords before the audio is rendered
• Zero-shot covers that reimagine a song transcribed from existing audio by SheetSage2
• Agentic music editing, where an existing track is refined through conversation rather than regenerated from scratch
• WildSongBench, a published 192-prompt evaluation set with reference scores and evaluators for reproducing the reported numbers
• Companion models for the full pipeline: YuE2-Vae decoding, SheetSage2 audio-to-score transcription and MERT-v2-FullSong representations

// Getting started

YuE2 installs as a Python package from its repository. It is documented for Linux with Python 3.12 and an NVIDIA GPU that supports BF16 and has 24 GB of VRAM; the team tested generation on an RTX 4090 and used H800 80GB cards for server deployments.

git clone https://github.com/multimodal-art-projection/YuE.git
cd YuE
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install .

python examples/generate.py --output outputs/first-song

import json
from pathlib import Path
from yue2 import YuE2Pipeline

request = json.loads(Path("examples/song.json").read_text(encoding="utf-8"))
with YuE2Pipeline.from_pretrained("m-a-p/YuE2-3B", device="cuda") as pipe:
    song = pipe(**request)
    song.save_artifacts("outputs/my-song")

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Research on full-song music generation, using WildSongBench to compare systems on a shared 192-prompt set
• Drafting a song from lyrics and a style prompt, then correcting the melody or chords in the score instead of rerolling the prompt
• Producing zero-shot covers of an existing track by transcribing it to a score and re-rendering it in another style
• Iterating on a generated track through agentic editing rather than regenerating it from the beginning

// How YuE2 compares

YuE2 alongside other open-source audio, music & voice tools AI/TLDR tracks, ranked by GitHub stars.