DExter: Learning and Controlling Performance Expression with Diffusion Models

TL;DR

DExter is a diffusion-based model that learns and controls expressive piano performances, enabling style transfer and perceptually guided variations, with promising quantitative and qualitative results.

Contribution

The paper introduces DExter, a novel diffusion model for expressive music performance rendering that incorporates perceptual features and enables style transfer.

Findings

Captures time-varying expressive parameters effectively

Performs well in subjective performance quality evaluations

Enables perceptually guided performance generation and style transfer

Abstract

In the pursuit of developing expressive music performance models using artificial intelligence, this paper introduces DExter, a new approach leveraging diffusion probabilistic models to render Western classical piano performances. In this approach, performance parameters are represented in a continuous expression space and a diffusion model is trained to predict these continuous parameters while being conditioned on the musical score. Furthermore, DExter also enables the generation of interpretations (expressive variations of a performance) guided by perceptually meaningful features by conditioning jointly on score and perceptual feature representations. Consequently, we find that our model is useful for learning expressive performance, generating perceptually steered performances, and transferring performance styles. We assess the model through quantitative and qualitative analyses, focusing on specific performance metrics regarding dimensions like asynchrony and articulation, as well as through listening tests comparing generated performances with different human interpretations. Results show that DExter is able to capture the time-varying correlation of the expressive parameters, and compares well to existing rendering models in subjectively evaluated ratings. The perceptual-feature-conditioned generation and transferring capabilities of DExter are verified by a proxy model predicting perceptual characteristics of differently steered performances.