MG-VAE: Deep Chinese Folk Songs Generation with Specific Regional Style

TL;DR

This paper introduces MG-VAE, a deep generative model that captures and manipulates regional styles in Chinese folk songs, enabling the creation of novel tunes with controllable style and content.

Contribution

It presents the first application of deep generative models with adversarial training for Chinese folk song generation, disentangling style, content, pitch, and rhythm in the latent space.

Findings

Successful disentanglement of style, content, pitch, and rhythm.

Ability to generate novel folk songs with specific regional styles.

First use of deep generative models for Chinese music creation.

Abstract

Regional style in Chinese folk songs is a rich treasure that can be used for ethnic music creation and folk culture research. In this paper, we propose MG-VAE, a music generative model based on VAE (Variational Auto-Encoder) that is capable of capturing specific music style and generating novel tunes for Chinese folk songs (Min Ge) in a manipulatable way. Specifically, we disentangle the latent space of VAE into four parts in an adversarial training way to control the information of pitch and rhythm sequence, as well as of music style and content. In detail, two classifiers are used to separate style and content latent space, and temporal supervision is utilized to disentangle the pitch and rhythm sequence. The experimental results show that the disentanglement is successful and our model is able to create novel folk songs with controllable regional styles. To our best knowledge, this is the first study on applying deep generative model and adversarial training for Chinese music generation.