Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation

TL;DR

This paper introduces a convolutional GAN model with binary neurons that directly generates binary-valued piano-rolls for polyphonic music, eliminating the need for post-processing and improving quality.

Contribution

It proposes a novel generator refiner network with binary neurons trained in two stages, achieving better binary music generation results than traditional thresholding methods.

Findings

Binary neurons outperform thresholding and Bernoulli sampling.

Deterministic binary neurons yield better subjective and objective results.

The model directly produces binary piano-rolls without post-processing.

Abstract

It has been shown recently that deep convolutional generative adversarial networks (GANs) can learn to generate music in the form of piano-rolls, which represent music by binary-valued time-pitch matrices. However, existing models can only generate real-valued piano-rolls and require further post-processing, such as hard thresholding (HT) or Bernoulli sampling (BS), to obtain the final binary-valued results. In this paper, we study whether we can have a convolutional GAN model that directly creates binary-valued piano-rolls by using binary neurons. Specifically, we propose to append to the generator an additional refiner network, which uses binary neurons at the output layer. The whole network is trained in two stages. Firstly, the generator and the discriminator are pretrained. Then, the refiner network is trained along with the discriminator to learn to binarize the real-valued piano-rolls the pretrained generator creates. Experimental results show that using binary neurons instead of HT or BS indeed leads to better results in a number of objective measures. Moreover, deterministic binary neurons perform better than stochastic ones in both objective measures and a subjective test. The source code, training data and audio examples of the generated results can be found at https://salu133445.github.io/bmusegan/ .