PIANOTREE VAE: Structured Representation Learning for Polyphonic Music

TL;DR

This paper introduces PianoTree VAE, a novel tree-structured variational autoencoder designed specifically for learning representations of complex polyphonic music, addressing limitations of previous monophonic-focused models.

Contribution

The paper presents a new tree-structured VAE model that effectively captures polyphonic music structures, enabling meaningful latent representations and improved music generation.

Findings

Latent codes are semantically meaningful for polyphonic segments

The model achieves better reconstruction quality

Enhances diversity in downstream music generation

Abstract

The dominant approach for music representation learning involves the deep unsupervised model family variational autoencoder (VAE). However, most, if not all, viable attempts on this problem have largely been limited to monophonic music. Normally composed of richer modality and more complex musical structures, the polyphonic counterpart has yet to be addressed in the context of music representation learning. In this work, we propose the PianoTree VAE, a novel tree-structure extension upon VAE aiming to fit the polyphonic music learning. The experiments prove the validity of the PianoTree VAE via (i)-semantically meaningful latent code for polyphonic segments; (ii)-more satisfiable reconstruction aside of decent geometry learned in the latent space; (iii)-this model's benefits to the variety of the downstream music generation.