SpaIn-Net: Spatially-Informed Stereophonic Music Source Separation

TL;DR

SpaIn-Net leverages explicit spatial information via panning angles to enhance stereophonic music source separation, enabling better instrument disentanglement and user interaction robustness.

Contribution

The paper introduces spatially-informed conditioning mechanisms for music source separation, improving performance and user control over traditional location-agnostic models.

Findings

Improved separation by 1.8 dB SI-SDR with spatial conditioning

Enabled disentanglement of same-class instruments

Robustness to incorrect panning information

Abstract

With the recent advancements of data driven approaches using deep neural networks, music source separation has been formulated as an instrument-specific supervised problem. While existing deep learning models implicitly absorb the spatial information conveyed by the multi-channel input signals, we argue that a more explicit and active use of spatial information could not only improve the separation process but also provide an entry-point for many user-interaction based tools. To this end, we introduce a control method based on the stereophonic location of the sources of interest, expressed as the panning angle. We present various conditioning mechanisms, including the use of raw angle and its derived feature representations, and show that spatial information helps. Our proposed approaches improve the separation performance compared to location agnostic architectures by 1.8 dB SI-SDR in our Slakh-based simulated experiments. Furthermore, the proposed methods allow for the disentanglement of same-class instruments, for example, in mixtures containing two guitar tracks. Finally, we also demonstrate that our approach is robust to incorrect source panning information, which can be incurred by our proposed user interaction.