Pre-trained Spatial Priors on Multichannel NMF for Music Source Separation

TL;DR

This paper introduces a pre-trained spatial prior integrated into multichannel NMF for improved music source separation, leveraging spatial information from recording setups to enhance separation quality.

Contribution

It proposes a novel pre-trained spatial filter incorporated into MNMF, utilizing room and transducer responses for better source separation in multichannel recordings.

Findings

Improved separation performance over conventional MNMF methods.

Effective in typical orchestra recording setups.

Applicable to existing recordings with standard microphone arrangements.

Abstract

This paper presents a novel approach to sound source separation that leverages spatial information obtained during the recording setup. Our method trains a spatial mixing filter using solo passages to capture information about the room impulse response and transducer response at each sensor location. This pre-trained filter is then integrated into a multichannel non-negative matrix factorization (MNMF) scheme to better capture the variances of different sound sources. The recording setup used in our experiments is the typical setup for orchestra recordings, with a main microphone and a close "cardioid" or "supercardioid" microphone for each section of the orchestra. This makes the proposed method applicable to many existing recordings. Experiments on polyphonic ensembles demonstrate the effectiveness of the proposed framework in separating individual sound sources, improving performance compared to conventional MNMF methods.