What is Dettato?
Dettato is KobiMusic's open-source audio-to-MIDI transcriber. Point it at a recording and it writes a MIDI file with one track per instrument and a separate drum track, plus a JSON file describing the notes. It sits in the Audio, Music & Voice category of the Multimodal Generation tools and is open-source, written in Python and licensed Apache-2.0.
Overview

Dettato is KobiMusic's open-source audio-to-MIDI transcriber. Point it at a recording and it writes a MIDI file with one track per instrument and a separate drum track, plus a JSON file describing the notes. The code and the model weights are released under Apache-2.0, and the weights download from the Hugging Face Hub the first time it runs.

Under the hood it chains four small models, 28.1M parameters in total: a 15.3M transcriber that finds the notes, a 4.9M evidence repairer that keeps, fixes or adds notes using the transcriber's probability maps and hidden state, a 2.9M grouper that assigns each note to an instrument track, and a 5.0M drum model for kit or orchestral percussion. One command runs the whole chain, on a GPU when one is available or on the CPU with `--device cpu`.

On the nine benchmark sets reported in the README, Dettato scores 59.2 composite F1 against 45.1 for the 1.3B-parameter MuScriptor. The project is open about its limits: pitched notes cover only the 88 piano keys, instrument naming is rough (on jazz, a quarter of some saxophone or lead guitar notes can land in the piano track), drums are skipped for audio recorded below 40 kHz, and the output is beat-aligned MIDI rather than sheet music.
What it does
- Multi-instrument transcription: one MIDI track per instrument plus a drum track, with a JSON file of the same notes
- A four-model pipeline of 28.1M parameters in total — transcriber, evidence repairer, grouper and drum model
- Command-line tool that takes files and folders (searched recursively) and writes `<name>.mid` and `<name>.json`
- Python API with `dettato.transcribe()` and a reusable `Dettato.from_pretrained()` model for batch jobs
- Weights fetched from Hugging Face on first use, or loaded from disk through the `DETTATO_WEIGHTS` folder
- Apache-2.0 code and weights; model cards list the license of each training dataset
Getting started
Dettato installs from GitHub with pip; the weights download from Hugging Face the first time you transcribe. The steps follow the project README.
Install and transcribe a song
Install straight from the repository, then pass an audio file and an output folder.
pip install git+https://github.com/kobimusic/dettato
dettato song.mp3 -o out/ # -> out/song.mid + out/song.jsonTranscribe several files or folders
Folders are searched recursively. A GPU is used when available; add `--device cpu` to force the CPU.
dettato song.mp3 other.wav folder/ -o out/Use it from Python
The result object exposes each instrument track with its General MIDI program, drum flag and notes.
import dettato
result = dettato.transcribe("song.mp3")
result.save_midi("song.mid")
for track in result.tracks:
print(track.name, track.program, track.is_drum, len(track.notes))Batch jobs: load the model once
For many files, load the models once and reuse them.
from dettato import Dettato
model = Dettato.from_pretrained("cuda")
for path in paths:
model.transcribe(path).save_midi(path + ".mid")Commands and code are distilled from the project's own documentation — always check the official repo for the latest.
When to use it
- Turn a band or orchestral recording into editable multi-track MIDI for a DAW
- Pull drum patterns out of a recording into their own MIDI track
- Batch-transcribe a folder of audio files into MIDI and JSON note data for analysis or datasets
- Add audio-to-MIDI to a music app without calling a paid API
How Dettato compares
Dettato alongside other open-source audio, music & voice tools AI/TLDR tracks, ranked by GitHub stars.
| Tool | Stars | What it does |
|---|---|---|
| VoiceStudio | ★ 116k | A local-first desktop voice studio that clones and designs voices, dubs video, and handles dictation, running 16 text-to-speech and 11 speech-recognition engines on your own hardware. |
| Whisper | ★ 113k | OpenAI's speech recognition model that transcribes and translates audio across many languages. |
| GPT-SoVITS | ★ 64.1k | An open-source WebUI that clones a voice from a short audio sample and turns text into speech, with zero-shot and few-shot fine-tuning. |
| Voicebox | ★ 61k | Local-first voice studio that clones a voice from a short sample, generates speech across seven TTS engines and 23 languages, handles system-wide dictation, and speaks for agents over MCP. |
| VibeVoice | ★ 57.4k | Microsoft's text-to-speech model for generating long, expressive multi-speaker audio like podcasts. |
| whisper.cpp | ★ 55.4k | A dependency-free C/C++ port of Whisper built on ggml, running speech recognition on CPU, Metal, CUDA, Vulkan and NPUs from phones to servers. |
| Coqui TTS | ★ 46.3k | A library of text-to-speech models including the multilingual XTTS voice-cloning model. |
| Dettato | ★ 175 | Open audio-to-MIDI transcription with one track per instrument, plus drums |
