Kyutai's MuScriptor Turns Any Band Recording Into Clean Per-Instrument MIDI
Kyutai and Mirelo release MuScriptor, an open-weight model that converts any multi-instrument recording into per-instrument MIDI using a novel three-stage training pipeline.
- MuScriptor is a new open-weight model from Kyutai and Mirelo that transcribes any multi-instrument recording into per-instrument MIDI.
- Trained in three stages: synthetic pre-training on 1.5M MIDI files, fine-tuning on 170k real recordings (11k hours), then RL post-training on 300 gold-standard tracks.
- Three model sizes: 103M (CPU-friendly), 307M (default), and 1.4B (GPU, highest accuracy).
- Supports instrument conditioning, streaming note events, and a local web UI with live piano roll; does not recover note velocity.
- Code is MIT licensed; model weights are CC BY-NC 4.0 (non-commercial). Try it at muscriptor.kyutai.org or install via
pip install muscriptor. - The paper identifies real-data fine-tuning as the single biggest quality driver, reframing the field's bottleneck from architecture to data.
Automatic music transcription has been a stubborn open problem for decades. Drop a full band recording into any existing tool, and you'll get back a muddy piano roll that blends all instruments into one indecipherable blob. MuScriptor, a new open-weight model from Kyutai and Mirelo, is the first serious attempt to crack this at scale: give it any recording, in any genre, and it returns clean, per-instrument MIDI.
The wall that stopped everyone else
Multi-instrument music transcription aims to convert polyphonic music recordings into musical scores assigned to each instrument. The task is challenging because it requires simultaneously identifying multiple instruments and transcribing their pitch and precise timing, and the lack of fully annotated data adds to the training difficulties.
The 2026 paper introduces MuScriptor as an open-weight multi-instrument music transcription model that works on real-world music recordings from across a diverse range of musical genres. The prior art, Google's MT3 from 2022, was a landmark in the field but hit a ceiling almost immediately: existing methods for automatic music transcription are often limited to single-instrument recordings or fail on complex, real music mixes. Although previous work utilizes synthetic training data, the resulting models generalize poorly, leading to largely unusable transcription output in realistic, multi-instrument settings.