Sequence-to-Sequence Piano Transcription with Transformers

TL;DR

This paper demonstrates that a generic encoder-decoder Transformer can effectively perform music transcription from spectrograms to MIDI-like outputs, simplifying the process and reducing the need for task-specific models.

Contribution

It introduces a sequence-to-sequence Transformer model for music transcription that achieves comparable performance to specialized models using standard decoding methods.

Findings

Transformer-based model achieves state-of-the-art transcription accuracy

Simplifies architecture by removing task-specific design

Enables direct translation from spectrograms to MIDI-like events

Abstract

Automatic Music Transcription has seen significant progress in recent years by training custom deep neural networks on large datasets. However, these models have required extensive domain-specific design of network architectures, input/output representations, and complex decoding schemes. In this work, we show that equivalent performance can be achieved using a generic encoder-decoder Transformer with standard decoding methods. We demonstrate that the model can learn to translate spectrogram inputs directly to MIDI-like output events for several transcription tasks. This sequence-to-sequence approach simplifies transcription by jointly modeling audio features and language-like output dependencies, thus removing the need for task-specific architectures. These results point toward possibilities for creating new Music Information Retrieval models by focusing on dataset creation and labeling rather than custom model design.