LEGATO: Large-scale End-to-end Generalizable Approach to Typeset OMR

TL;DR

Legato is a large-scale, end-to-end pretrained model for optical music recognition that effectively converts music score images into ABC notation, demonstrating strong generalization and superior performance over previous methods.

Contribution

This paper introduces Legato, the first large-scale pretrained OMR model capable of recognizing full-page scores and generating ABC notation, with extensive experiments showing its effectiveness.

Findings

Achieves 68% error reduction on TEDn metric

Achieves 47.6% error reduction on OMR-NED metric

Outperforms previous state-of-the-art models

Abstract

We propose Legato, a new end-to-end model for optical music recognition (OMR), a task of converting music score images to machine-readable documents. Legato is the first large-scale pretrained OMR model capable of recognizing full-page or multi-page typeset music scores and the first to generate documents in ABC notation, a concise, human-readable format for symbolic music. Bringing together a pretrained vision encoder with an ABC decoder trained on a dataset of more than 214K images, our model exhibits the strong ability to generalize across various typeset scores. We conduct comprehensive experiments on a range of datasets and metrics and demonstrate that Legato outperforms the previous state of the art. On our most realistic dataset, we see a 68\% and 47.6\% absolute error reduction on the standard metrics TEDn and OMR-NED, respectively.