Generating Lead Sheets with Affect: A Novel Conditional seq2seq Framework

TL;DR

This paper introduces a novel conditional sequence-to-sequence framework for automatic lead sheet music generation that incorporates high-level emotional and structural controls like valence, phrasing, and time signature, using neural networks.

Contribution

It presents a new method for calculating emotional valence of chord progressions and integrates this into a neural machine translation model for controllable music generation.

Findings

Controllable lead sheet generation with similar attribute distributions to training data.

Effective control of valence confirmed through listening tests.

Comparison of LSTM and Transformer architectures for music generation.

Abstract

The field of automatic music composition has seen great progress in the last few years, much of which can be attributed to advances in deep neural networks. There are numerous studies that present different strategies for generating sheet music from scratch. The inclusion of high-level musical characteristics (e.g., perceived emotional qualities), however, as conditions for controlling the generation output remains a challenge. In this paper, we present a novel approach for calculating the valence (the positivity or negativity of the perceived emotion) of a chord progression within a lead sheet, using pre-defined mood tags proposed by music experts. Based on this approach, we propose a novel strategy for conditional lead sheet generation that allows us to steer the music generation in terms of valence, phrasing, and time signature. Our approach is similar to a Neural Machine Translation (NMT) problem, as we include high-level conditions in the encoder part of the sequence-to-sequence architectures used (i.e., long-short term memory networks, and a Transformer network). We conducted experiments to thoroughly analyze these two architectures. The results show that the proposed strategy is able to generate lead sheets in a controllable manner, resulting in distributions of musical attributes similar to those of the training dataset. We also verified through a subjective listening test that our approach is effective in controlling the valence of a generated chord progression.