Kyutai's MuScriptor Doubles Open-Source Music Transcription Accuracy to 48.2 F1
Kyutai and Mirelo's open transcription model now exports editable sheet music and guitar tabs, on top of streaming per-instrument MIDI.
PRO- MuScriptor now exports sheet music and guitar tabs as PDF and editable MusicXML, generated from MIDI via MuseScore.
- Open-weight decoder-only Transformer from Kyutai and Mirelo for multi-instrument music transcription.
- Three sizes: 103M, 307M, and 1.4B parameters, auto-downloaded from Hugging Face on first use.
- Trained on 1.45M synthetic MIDI plus 170k real recordings (11k+ hours) with RL post-training.
- Large variant scores Multi F1 48.2 versus 21.9 for YourMT3+ baseline; try it at muscriptor.kyutai.org.
- Code is MIT-licensed, weights are CC BY-NC 4.0 (non-commercial); works best on steady-tempo music.
Automatic music transcription has been chronically half-solved. Drop a full band mix into most tools and you get back a smeared piano roll that fuses drums, bass, guitar, and vocals into one unreadable blob. MuScriptor, a joint model from Kyutai and Mirelo, was released to fix that, and it just got a practical upgrade: sheet music and guitar tabs, not only MIDI.
The pipeline exports both PDFs and editable MusicXML, so output drops straight into notation software without hand re-engraving. Under the hood, the team generates scores from MIDI using MuseScore plus custom preprocessing to clean up rhythm and layout. Everything runs from the same open-source repo, and you can try it in the browser at muscriptor.kyutai.org or self-host locally.
What's actually under the hood
MuScriptor is a decoder-only Transformer for music transcription. It reads a mel-spectrogram of a short audio segment, then autoregressively predicts MIDI-like tokens for pitch, timing, and instrument, turning transcription into a language-modeling task following the MT3 tokenization scheme. Feed it a recording in any genre (pop, classical, metal, jazz) and it transcribes the notes played by every instrument into MIDI, without needing to know in advance what instruments are present.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
