Feature-informed Latent Space Regularization for Music Source Separation

TL;DR

This paper proposes transfer learning strategies that incorporate VGGish features into a music source separation model, improving performance without requiring additional annotations during training or inference.

Contribution

It introduces three novel approaches, including latent space regularization methods, to effectively integrate VGGish features into source separation models.

Findings

Improved evaluation metrics for music source separation.

Latent space regularization outperforms naive concatenation.

VGGish features enhance separation quality.

Abstract

The integration of additional side information to improve music source separation has been investigated numerous times, e.g., by adding features to the input or by adding learning targets in a multi-task learning scenario. These approaches, however, require additional annotations such as musical scores, instrument labels, etc. in training and possibly during inference. The available datasets for source separation do not usually provide these additional annotations. In this work, we explore transfer learning strategies to incorporate VGGish features with a state-of-the-art source separation model; VGGish features are known to be a very condensed representation of audio content and have been successfully used in many MIR tasks. We introduce three approaches to incorporate the features, including two latent space regularization methods and one naive concatenation method. Experimental results show that our proposed approaches improve several evaluation metrics for music source separation.