Checklist Models for Improved Output Fluency in Piano Fingering Prediction

TL;DR

This paper introduces a reinforcement learning-based checklist model for piano fingering prediction that improves output fluency and human performability by considering output structure and relative keyboard position.

Contribution

It proposes a novel checklist system trained with reinforcement learning and a new input encoding scheme, enhancing fluency and human-like performance in piano fingering prediction.

Findings

Enhanced fluency metrics correlate with human performability.

Reinforcement learning reduces physically challenging finger sequences.

Modified input encoding improves prediction accuracy.

Abstract

In this work we present a new approach for the task of predicting fingerings for piano music. While prior neural approaches have often treated this as a sequence tagging problem with independent predictions, we put forward a checklist system, trained via reinforcement learning, that maintains a representation of recent predictions in addition to a hidden state, allowing it to learn soft constraints on output structure. We also demonstrate that by modifying input representations -- which in prior work using neural models have often taken the form of one-hot encodings over individual keys on the piano -- to encode relative position on the keyboard to the prior note instead, we can achieve much better performance. Additionally, we reassess the use of raw per-note labeling precision as an evaluation metric, noting that it does not adequately measure the fluency, i.e. human playability, of a model's output. To this end, we compare methods across several statistics which track the frequency of adjacent finger predictions that while independently reasonable would be physically challenging to perform in sequence, and implement a reinforcement learning strategy to minimize these as part of our training loss. Finally through human expert evaluation, we demonstrate significant gains in performability directly attributable to improvements with respect to these metrics.