Real-Time Audio-to-Score Alignment of Music Performances Containing Errors and Arbitrary Repeats and Skips

TL;DR

This paper introduces efficient real-time algorithms for audio-to-score alignment that handle errors, tempo changes, and arbitrary repeats/skips in music performances, enabling practical automatic accompaniment systems.

Contribution

It proposes two hidden Markov models and algorithms that efficiently align monophonic performances with complex errors and repeats, suitable for scores of practical length.

Findings

Real-time alignment of scores with ~10,000 notes achieved on a laptop.

Tracking within 0.7 seconds after repeats/skips in clarinet performances.

Algorithms extend previous methods to handle errors and arbitrary repeats/skips efficiently.

Abstract

This paper discusses real-time alignment of audio signals of music performance to the corresponding score (a.k.a. score following) which can handle tempo changes, errors and arbitrary repeats and/or skips (repeats/skips) in performances. This type of score following is particularly useful in automatic accompaniment for practices and rehearsals, where errors and repeats/skips are often made. Simple extensions of the algorithms previously proposed in the literature are not applicable in these situations for scores of practical length due to the problem of large computational complexity. To cope with this problem, we present two hidden Markov models of monophonic performance with errors and arbitrary repeats/skips, and derive efficient score-following algorithms with an assumption that the prior probability distributions of score positions before and after repeats/skips are independent from each other. We confirmed real-time operation of the algorithms with music scores of practical length (around 10000 notes) on a modern laptop and their tracking ability to the input performance within 0.7 s on average after repeats/skips in clarinet performance data. Further improvements and extension for polyphonic signals are also discussed.