# Dettato

> Open audio-to-MIDI transcription with one track per instrument, plus drums

- Category: Multimodal Generation / Audio, Music & Voice
- Language: Python
- Licence: Apache-2.0
- Repository: https://github.com/kobimusic/dettato
- Website: https://kobi.music/research/dettato
- Canonical page: https://ai-tldr.dev/tools/dettato/
- Updated: 2026-11-16

## In short

Dettato is KobiMusic's open-source audio-to-MIDI transcriber. Point it at a recording and it writes a MIDI file with one track per instrument and a separate drum track, plus a JSON file describing the notes. It sits in the Audio, Music & Voice category of the Multimodal Generation tools and is open-source, written in Python and licensed Apache-2.0.

## Overview

Dettato is KobiMusic's open-source audio-to-MIDI transcriber. Point it at a recording and it writes a MIDI file with one track per instrument and a separate drum track, plus a JSON file describing the notes. The code and the model weights are released under Apache-2.0, and the weights download from the Hugging Face Hub the first time it runs.

Under the hood it chains four small models, 28.1M parameters in total: a 15.3M transcriber that finds the notes, a 4.9M evidence repairer that keeps, fixes or adds notes using the transcriber's probability maps and hidden state, a 2.9M grouper that assigns each note to an instrument track, and a 5.0M drum model for kit or orchestral percussion. One command runs the whole chain, on a GPU when one is available or on the CPU with `--device cpu`.

On the nine benchmark sets reported in the README, Dettato scores 59.2 composite F1 against 45.1 for the 1.3B-parameter MuScriptor. The project is open about its limits: pitched notes cover only the 88 piano keys, instrument naming is rough (on jazz, a quarter of some saxophone or lead guitar notes can land in the piano track), drums are skipped for audio recorded below 40 kHz, and the output is beat-aligned MIDI rather than sheet music.

## What it does

- Multi-instrument transcription: one MIDI track per instrument plus a drum track, with a JSON file of the same notes
- A four-model pipeline of 28.1M parameters in total — transcriber, evidence repairer, grouper and drum model
- Command-line tool that takes files and folders (searched recursively) and writes `<name>.mid` and `<name>.json`
- Python API with `dettato.transcribe()` and a reusable `Dettato.from_pretrained()` model for batch jobs
- Weights fetched from Hugging Face on first use, or loaded from disk through the `DETTATO_WEIGHTS` folder
- Apache-2.0 code and weights; model cards list the license of each training dataset

## Getting started

Dettato installs from GitHub with pip; the weights download from Hugging Face the first time you transcribe. The steps follow the project README.

1. Install and transcribe a song

   Install straight from the repository, then pass an audio file and an output folder.

```bash
pip install git+https://github.com/kobimusic/dettato
dettato song.mp3 -o out/        # -> out/song.mid + out/song.json
```

2. Transcribe several files or folders

   Folders are searched recursively. A GPU is used when available; add `--device cpu` to force the CPU.

```bash
dettato song.mp3 other.wav folder/ -o out/
```

3. Use it from Python

   The result object exposes each instrument track with its General MIDI program, drum flag and notes.

```python
import dettato

result = dettato.transcribe("song.mp3")
result.save_midi("song.mid")
for track in result.tracks:
    print(track.name, track.program, track.is_drum, len(track.notes))
```

4. Batch jobs: load the model once

   For many files, load the models once and reuse them.

```python
from dettato import Dettato

model = Dettato.from_pretrained("cuda")
for path in paths:
    model.transcribe(path).save_midi(path + ".mid")
```

## When to use it

- Turn a band or orchestral recording into editable multi-track MIDI for a DAW
- Pull drum patterns out of a recording into their own MIDI track
- Batch-transcribe a folder of audio files into MIDI and JSON note data for analysis or datasets
- Add audio-to-MIDI to a music app without calling a paid API

## Version history

- 2026-11-16: First public release: code and weights for the four-model audio-to-MIDI pipeline, with a browser demo on KobiMusic's research page.
