Count The Notes: Histogram-Based Supervision for Automatic Music Transcription

TL;DR

CountEM introduces a histogram-based supervision method for automatic music transcription that removes the need for local alignment, reducing annotation effort while maintaining high accuracy across multiple instruments.

Contribution

It presents CountEM, a novel EM-based framework that uses note occurrence histograms for weakly supervised AMT, eliminating local alignment requirements and improving efficiency.

Findings

CountEM achieves comparable or better accuracy than existing weakly supervised methods.

The approach reduces annotation effort by relying solely on note counts.

Experiments show improved robustness and scalability across diverse datasets.

Abstract

Automatic Music Transcription (AMT) converts audio recordings into symbolic musical representations. Training deep neural networks (DNNs) for AMT typically requires strongly aligned training pairs with precise frame-level annotations. Since creating such datasets is costly and impractical for many musical contexts, weakly aligned approaches using segment-level annotations have gained traction. However, existing methods often rely on Dynamic Time Warping (DTW) or soft alignment loss functions, both of which still require local semantic correspondences, making them error-prone and computationally expensive. In this article, we introduce CountEM, a novel AMT framework that eliminates the need for explicit local alignment by leveraging note event histograms as supervision, enabling lighter computations and greater flexibility. Using an Expectation-Maximization (EM) approach, CountEM iteratively refines predictions based solely on note occurrence counts, significantly reducing annotation efforts while maintaining high transcription accuracy. Experiments on piano, guitar, and multi-instrument datasets demonstrate that CountEM matches or surpasses existing weakly supervised methods, improving AMT's robustness, scalability, and efficiency. Our project page is available at https://yoni-yaffe.github.io/count-the-notes.