PocketVAE: A Two-step Model for Groove Generation and Control

TL;DR

PocketVAE is a two-step groove generation system that enhances MIDI drum tracks by transferring, generating, and controlling grooves using a discrete latent space, improving the modeling of drum note distributions.

Contribution

The paper introduces a novel two-step approach with discrete latent representations for groove generation and control in MIDI drum tracks, combining note editing with velocity and microtiming details.

Findings

Discrete latent space improves data distribution modeling.

Two-step approach enhances groove realism.

Incorporating control elements enables customizable groove generation.

Abstract

Creating a good drum track to imitate a skilled performer in digital audio workstations (DAWs) can be a time-consuming process, especially for those unfamiliar with drums. In this work, we introduce PocketVAE, a groove generation system that applies grooves to users' rudimentary MIDI tracks, i.e, templates. Grooves can be either transferred from a reference track, generated randomly or with conditions, such as genres. Our system, consisting of different modules for each groove component, takes a two-step approach that is analogous to a music creation process. First, the note module updates the user template through addition and deletion of notes; Second, the velocity and microtiming modules add details to this generated note score. In order to model the drum notes, we apply a discrete latent representation method via Vector Quantized Variational Autoencoder (VQ-VAE), as drum notes have a discrete property, unlike velocity and microtiming values. We show that our two-step approach and the usage of a discrete encoding space improves the learning of the original data distribution. Additionally, we discuss the benefit of incorporating control elements - genre, velocity and microtiming patterns - into the model.