The Inverse Drum Machine: Source Separation Through Joint Transcription and Analysis-by-Synthesis

TL;DR

The paper introduces the Inverse Drum Machine, a novel source separation method that combines analysis-by-synthesis with deep learning, trained on mixtures with transcription annotations, achieving state-of-the-art results without needing isolated stems.

Contribution

It presents a new joint transcription and synthesis framework for drum source separation that does not require isolated drum stems for training.

Findings

Achieves separation quality comparable to supervised methods.

Operates effectively with only transcription annotations.

Demonstrates robustness on the StemGMD dataset.

Abstract

We present the Inverse Drum Machine, a novel approach to Drum Source Separation that leverages an analysis-by-synthesis framework combined with deep learning. Unlike recent supervised methods that require isolated stem recordings for training, our approach is trained on drum mixtures with only transcription annotations. IDM integrates Automatic Drum Transcription and One-shot Drum Sample Synthesis, jointly optimizing these tasks in an end-to-end manner. By convolving synthesized one-shot samples with estimated onsets, akin to a drum machine, we reconstruct the individual drum stems and train a Deep Neural Network on the reconstruction of the mixture. Experiments on the StemGMD dataset demonstrate that IDM achieves separation quality comparable to state-of-the-art supervised methods that require isolated stems data.