Generating music with sentiment using Transformer-GANs

TL;DR

This paper introduces a Transformer-GAN model for symbolic music generation conditioned on human sentiment, enabling more interactive and emotionally coherent music synthesis with improved long-term structure.

Contribution

It presents a novel Transformer-GAN architecture that incorporates sentiment conditioning and efficient attention mechanisms for improved music generation.

Findings

Enhanced coherence in long-term music sequences

Effective incorporation of human sentiment into generated music

Improved quality of music through discriminator feedback

Abstract

The field of Automatic Music Generation has seen significant progress thanks to the advent of Deep Learning. However, most of these results have been produced by unconditional models, which lack the ability to interact with their users, not allowing them to guide the generative process in meaningful and practical ways. Moreover, synthesizing music that remains coherent across longer timescales while still capturing the local aspects that make it sound ``realistic'' or ``human-like'' is still challenging. This is due to the large computational requirements needed to work with long sequences of data, and also to limitations imposed by the training schemes that are often employed. In this paper, we propose a generative model of symbolic music conditioned by data retrieved from human sentiment. The model is a Transformer-GAN trained with labels that correspond to different configurations of the valence and arousal dimensions that quantitatively represent human affective states. We try to tackle both of the problems above by employing an efficient linear version of Attention and using a Discriminator both as a tool to improve the overall quality of the generated music and its ability to follow the conditioning signals.