Exploring Classical Piano Performance Generation with Expressive Music Variational AutoEncoder
Jing Luo, Xinyu Yang, and Jie Wei

TL;DR
This paper introduces XMVAE, a novel model that generates expressive classical piano performances by jointly modeling score content and expressive nuances, achieving superior quality through a dual-branch VAE architecture.
Contribution
The paper presents the Expressive Music Variational AutoEncoder (XMVAE), combining VQ-VAE and VAE branches with multiscale encoding for improved expressive performance generation.
Findings
XMVAE outperforms state-of-the-art models in quality.
Pretraining on score datasets enhances performance.
Objective and subjective evaluations confirm effectiveness.
Abstract
The creativity of classical music arises not only from composers who craft the musical sheets but also from performers who interpret the static notations with expressive nuances. This paper addresses the challenge of generating classical piano performances from scratch, aiming to emulate the dual roles of composer and pianist in the creative process. We introduce the Expressive Compound Word (ECP) representation, which effectively captures both the metrical structure and expressive nuances of classical performances. Building on this, we propose the Expressive Music Variational AutoEncoder (XMVAE), a model featuring two branches: a Vector Quantized Variational AutoEncoder (VQ-VAE) branch that generates score-related content, representing the Composer, and a vanilla VAE branch that produces expressive details, fulfilling the role of Pianist. These branches are jointly trained with similar…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMusic Technology and Sound Studies · Music and Audio Processing · Neuroscience and Music Perception
