Sheet Music Benchmark: Standardized Optical Music Recognition Evaluation

TL;DR

This paper introduces the Sheet Music Benchmark (SMB), a comprehensive dataset for evaluating Optical Music Recognition (OMR), along with a new evaluation metric, OMR-NED, to improve accuracy and comparability in OMR research.

Contribution

The paper presents the SMB dataset and the OMR-NED metric, providing standardized tools for benchmarking and detailed error analysis in OMR research.

Findings

Baseline experiments demonstrate the effectiveness of SMB and OMR-NED.

OMR-NED offers more detailed error insights than traditional metrics.

SMB covers diverse musical textures for comprehensive evaluation.

Abstract

In this work, we introduce the Sheet Music Benchmark (SMB), a dataset of six hundred and eighty-five pages specifically designed to benchmark Optical Music Recognition (OMR) research. SMB encompasses a diverse array of musical textures, including monophony, pianoform, quartet, and others, all encoded in Common Western Modern Notation using the Humdrum **kern format. Alongside SMB, we introduce the OMR Normalized Edit Distance (OMR-NED), a new metric tailored explicitly for evaluating OMR performance. OMR-NED builds upon the widely-used Symbol Error Rate (SER), offering a fine-grained and detailed error analysis that covers individual musical elements such as note heads, beams, pitches, accidentals, and other critical notation features. The resulting numeric score provided by OMR-NED facilitates clear comparisons, enabling researchers and end-users alike to identify optimal OMR approaches. Our work thus addresses a long-standing gap in OMR evaluation, and we support our contributions with baseline experiments using standardized SMB dataset splits for training and assessing state-of-the-art methods.