Analyzing and reducing the synthetic-to-real transfer gap in Music Information Retrieval: the task of automatic drum transcription

TL;DR

This paper investigates methods to improve the realism of synthetic datasets for automatic drum transcription, demonstrating strategies that reduce the transfer gap between synthetic and real data, and analyzing the limits of data quantity in model training.

Contribution

It introduces three strategies to enhance synthetic data realism and evaluates their effectiveness in narrowing the transfer gap in drum transcription models.

Findings

Strategies improve synthetic data realism and transfer performance.

The new dataset has the most realistic data distribution among evaluated datasets.

Limits of infinite data training are identified and addressed.

Abstract

Automatic drum transcription is a critical tool in Music Information Retrieval for extracting and analyzing the rhythm of a music track, but it is limited by the size of the datasets available for training. A popular method used to increase the amount of data is by generating them synthetically from music scores rendered with virtual instruments. This method can produce a virtually infinite quantity of tracks, but empirical evidence shows that models trained on previously created synthetic datasets do not transfer well to real tracks. In this work, besides increasing the amount of data, we identify and evaluate three more strategies that practitioners can use to improve the realism of the generated data and, thus, narrow the synthetic-to-real transfer gap. To explore their efficacy, we used them to build a new synthetic dataset and then we measured how the performance of a model scales and, specifically, at what value it will stagnate when increasing the number of training tracks for different datasets. By doing this, we were able to prove that the aforementioned strategies contribute to make our dataset the one with the most realistic data distribution and the lowest synthetic-to-real transfer gap among the synthetic datasets we evaluated. We conclude by highlighting the limits of training with infinite data in drum transcription and we show how they can be overcome.