SyMuPe: Affective and Controllable Symbolic Music Performance

TL;DR

SyMuPe introduces a novel affective and controllable symbolic piano performance framework using conditional flow matching, enabling expressive, emotion-aware music generation that rivals human performances and supports interactive applications.

Contribution

The paper presents PianoFlow, a new conditional flow matching model for expressive, emotion-controlled symbolic piano performance, trained on a large dataset and outperforming existing transformer-based models.

Findings

PianoFlow outperforms transformer baselines in quality.

The model achieves human-like performance levels.

Emotion control via text conditioning is effective.

Abstract

Emotions are fundamental to the creation and perception of music performances. However, achieving human-like expression and emotion through machine learning models for performance rendering remains a challenging task. In this work, we present SyMuPe, a novel framework for developing and training affective and controllable symbolic piano performance models. Our flagship model, PianoFlow, uses conditional flow matching trained to solve diverse multi-mask performance inpainting tasks. By design, it supports both unconditional generation and infilling of music performance features. For training, we use a curated, cleaned dataset of 2,968 hours of aligned musical scores and expressive MIDI performances. For text and emotion control, we integrate a piano performance emotion classifier and tune PianoFlow with the emotion-weighted Flan-T5 text embeddings provided as conditional inputs. Objective and subjective evaluations against transformer-based baselines and existing models show that PianoFlow not only outperforms other approaches, but also achieves performance quality comparable to that of human-recorded and transcribed MIDI samples. For emotion control, we present and analyze samples generated under different text conditioning scenarios. The developed model can be integrated into interactive applications, contributing to the creation of more accessible and engaging music performance systems.