█

AI/TLDR

Buy me a coffee
Buy me a coffee

Dettato

Open audio-to-MIDI transcription with one track per instrument, plus drums

Audio, Music & VoiceOpen source
Updated
16 Nov 2026
Language
Python
License
Apache-2.0
Coverage
1 story
$pip install git+https://github.com/kobimusic/dettato

What's new

16 Nov 2026

First public release: code and weights for the four-model audio-to-MIDI pipeline, with a browser demo on KobiMusic's research page.

Latest news

IN SHORT

What is Dettato?

Dettato is KobiMusic's open-source audio-to-MIDI transcriber. Point it at a recording and it writes a MIDI file with one track per instrument and a separate drum track, plus a JSON file describing the notes. It sits in the Audio, Music & Voice category of the Multimodal Generation tools and is open-source, written in Python and licensed Apache-2.0.

Updated Markdown version ↗

Overview

Dettato's transcription of Beethoven's Symphony No. 5 split into separate instrument tracks of notes
Example output: Beethoven's Symphony No. 5 transcribed into per-instrument MIDI tracks.Dettato README ↗

Dettato is KobiMusic's open-source audio-to-MIDI transcriber. Point it at a recording and it writes a MIDI file with one track per instrument and a separate drum track, plus a JSON file describing the notes. The code and the model weights are released under Apache-2.0, and the weights download from the Hugging Face Hub the first time it runs.

Diagram of Dettato's pipeline: audio goes to a transcriber, evidence repairer and grouper, with a separate drum model, into a multi-track MIDI file
Four models, 28.1M parameters in all; one command runs them.Dettato README ↗

Under the hood it chains four small models, 28.1M parameters in total: a 15.3M transcriber that finds the notes, a 4.9M evidence repairer that keeps, fixes or adds notes using the transcriber's probability maps and hidden state, a 2.9M grouper that assigns each note to an instrument track, and a 5.0M drum model for kit or orchestral percussion. One command runs the whole chain, on a GPU when one is available or on the CPU with `--device cpu`.

Bar chart and table: Dettato scores 59.2 composite F1 across nine benchmark sets versus 45.1 for MuScriptor
Composite F1 on nine benchmark sets, as reported by the project.Dettato README ↗

On the nine benchmark sets reported in the README, Dettato scores 59.2 composite F1 against 45.1 for the 1.3B-parameter MuScriptor. The project is open about its limits: pitched notes cover only the 88 piano keys, instrument naming is rough (on jazz, a quarter of some saxophone or lead guitar notes can land in the piano track), drums are skipped for audio recorded below 40 kHz, and the output is beat-aligned MIDI rather than sheet music.

What it does

  • Multi-instrument transcription: one MIDI track per instrument plus a drum track, with a JSON file of the same notes
  • A four-model pipeline of 28.1M parameters in total — transcriber, evidence repairer, grouper and drum model
  • Command-line tool that takes files and folders (searched recursively) and writes `<name>.mid` and `<name>.json`
  • Python API with `dettato.transcribe()` and a reusable `Dettato.from_pretrained()` model for batch jobs
  • Weights fetched from Hugging Face on first use, or loaded from disk through the `DETTATO_WEIGHTS` folder
  • Apache-2.0 code and weights; model cards list the license of each training dataset

Getting started

Dettato installs from GitHub with pip; the weights download from Hugging Face the first time you transcribe. The steps follow the project README.

Install and transcribe a song

Install straight from the repository, then pass an audio file and an output folder.

bashbash
pip install git+https://github.com/kobimusic/dettato
dettato song.mp3 -o out/        # -> out/song.mid + out/song.json

Transcribe several files or folders

Folders are searched recursively. A GPU is used when available; add `--device cpu` to force the CPU.

bashbash
dettato song.mp3 other.wav folder/ -o out/

Use it from Python

The result object exposes each instrument track with its General MIDI program, drum flag and notes.

pythonpython
import dettato

result = dettato.transcribe("song.mp3")
result.save_midi("song.mid")
for track in result.tracks:
    print(track.name, track.program, track.is_drum, len(track.notes))

Batch jobs: load the model once

For many files, load the models once and reuse them.

pythonpython
from dettato import Dettato

model = Dettato.from_pretrained("cuda")
for path in paths:
    model.transcribe(path).save_midi(path + ".mid")

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

When to use it

  • Turn a band or orchestral recording into editable multi-track MIDI for a DAW
  • Pull drum patterns out of a recording into their own MIDI track
  • Batch-transcribe a folder of audio files into MIDI and JSON note data for analysis or datasets
  • Add audio-to-MIDI to a music app without calling a paid API

How Dettato compares

Dettato alongside other open-source audio, music & voice tools AI/TLDR tracks, ranked by GitHub stars.

ToolStarsWhat it does
VoiceStudio★ 116kA local-first desktop voice studio that clones and designs voices, dubs video, and handles dictation, running 16 text-to-speech and 11 speech-recognition engines on your own hardware.
Whisper★ 113kOpenAI's speech recognition model that transcribes and translates audio across many languages.
GPT-SoVITS★ 64.1kAn open-source WebUI that clones a voice from a short audio sample and turns text into speech, with zero-shot and few-shot fine-tuning.
Voicebox★ 61kLocal-first voice studio that clones a voice from a short sample, generates speech across seven TTS engines and 23 languages, handles system-wide dictation, and speaks for agents over MCP.
VibeVoice★ 57.4kMicrosoft's text-to-speech model for generating long, expressive multi-speaker audio like podcasts.
whisper.cpp★ 55.4kA dependency-free C/C++ port of Whisper built on ggml, running speech recognition on CPU, Metal, CUDA, Vulkan and NPUs from phones to servers.
Coqui TTS★ 46.3kA library of text-to-speech models including the multilingual XTTS voice-cloning model.
Dettato★ 175Open audio-to-MIDI transcription with one track per instrument, plus drums