Generating Coherent Drum Accompaniment With Fills And Improvisations

TL;DR

This paper presents a transformer-based approach for generating coherent drum accompaniments with fills and improvisations conditioned on melodic instruments, introducing a novelty function and in-filling architecture to enhance improvisation modeling.

Contribution

It introduces a novel BERT-inspired in-filling architecture and a new improvisation detection function for improved drum pattern generation conditioned on melodic accompaniment.

Findings

Transformer model captures basic drum patterns conditioned on melody.

Proposed novelty function effectively identifies improvisation segments.

In-filling architecture improves the integration of improvisations into drum patterns.

Abstract

Creating a complex work of art like music necessitates profound creativity. With recent advancements in deep learning and powerful models such as transformers, there has been huge progress in automatic music generation. In an accompaniment generation context, creating a coherent drum pattern with apposite fills and improvisations at proper locations in a song is a challenging task even for an experienced drummer. Drum beats tend to follow a repetitive pattern through stanzas with fills or improvisation at section boundaries. In this work, we tackle the task of drum pattern generation conditioned on the accompanying music played by four melodic instruments: Piano, Guitar, Bass, and Strings. We use the transformer sequence to sequence model to generate a basic drum pattern conditioned on the melodic accompaniment to find that improvisation is largely absent, attributed possibly to its expectedly relatively low representation in the training data. We propose a novelty function to capture the extent of improvisation in a bar relative to its neighbors. We train a model to predict improvisation locations from the melodic accompaniment tracks. Finally, we use a novel BERT-inspired in-filling architecture, to learn the structure of both the drums and melody to in-fill elements of improvised music.