Audio query-based music source separation

TL;DR

This paper introduces an audio query-based music source separation network that can separate multiple sources from a mixture using a query signal, with the ability to generate continuous outputs through latent space interpolation.

Contribution

It proposes a novel network architecture with a Query-net and Separator that explicitly encodes source information from a query, enabling flexible separation of multiple sources.

Findings

Successfully separates multiple sources with a single network.

Can generate continuous outputs via latent vector interpolation.

Performs well on the MUSDB18 dataset.

Abstract

In recent years, music source separation has been one of the most intensively studied research areas in music information retrieval. Improvements in deep learning lead to a big progress in music source separation performance. However, most of the previous studies are restricted to separating a few limited number of sources, such as vocals, drums, bass, and other. In this study, we propose a network for audio query-based music source separation that can explicitly encode the source information from a query signal regardless of the number and/or kind of target signals. The proposed method consists of a Query-net and a Separator: given a query and a mixture, the Query-net encodes the query into the latent space, and the Separator estimates masks conditioned by the latent vector, which is then applied to the mixture for separation. The Separator can also generate masks using the latent vector from the training samples, allowing separation in the absence of a query. We evaluate our method on the MUSDB18 dataset, and experimental results show that the proposed method can separate multiple sources with a single network. In addition, through further investigation of the latent space we demonstrate that our method can generate continuous outputs via latent vector interpolation.