End-to-End Full-Page Optical Music Recognition for Pianoform Sheet Music

TL;DR

This paper introduces the first fully end-to-end system for page-level optical music recognition that processes entire scores with a combined convolutional and Transformer architecture, outperforming existing commercial tools.

Contribution

The paper presents a novel end-to-end OMR approach that eliminates multi-stage pipelines, using curriculum learning and synthetic data to handle complex music score layouts.

Findings

Successfully transcribes full-page scores in synthetic and real-world data

Outperforms commercial OMR software in zero-shot and fine-tuned scenarios

Demonstrates robustness on complex pianoform corpora

Abstract

Optical Music Recognition (OMR) has made significant progress since its inception, with various approaches now capable of accurately transcribing music scores into digital formats. Despite these advancements, most so-called end-to-end OMR approaches still rely on multi-stage processing pipelines for transcribing full-page score images, which entails challenges such as the need for dedicated layout analysis and specific annotated data, thereby limiting the general applicability of such methods. In this paper, we present the first truly end-to-end approach for page-level OMR in complex layouts. Our system, which combines convolutional layers with autoregressive Transformers, processes an entire music score page and outputs a complete transcription in a music encoding format. This is made possible by both the architecture and the training procedure, which utilizes curriculum learning through incremental synthetic data generation. We evaluate the proposed system using pianoform corpora, which is one of the most complex sources in the OMR literature. This evaluation is conducted first in a controlled scenario with synthetic data, and subsequently against two real-world corpora of varying conditions. Our approach is compared with leading commercial OMR software. The results demonstrate that our system not only successfully transcribes full-page music scores but also outperforms the commercial tool in both zero-shot settings and after fine-tuning with the target domain, representing a significant contribution to the field of OMR.