UNMIXX: Untangling Highly Correlated Singing Voices Mixtures

TL;DR

UNMIXX is a new framework for separating highly correlated singing voices, overcoming data scarcity and correlation challenges with innovative training strategies and attention mechanisms, significantly improving separation quality.

Contribution

It introduces a musically informed mixing strategy, cross-source attention, and a magnitude penalty loss to enhance singing voices separation in highly correlated mixtures.

Findings

Achieves over 2.2 dB SDRi improvement over previous methods.

Effectively handles highly correlated singing voice mixtures.

Addresses data scarcity with realistic training data simulation.

Abstract

We introduce UNMIXX, a novel framework for multiple singing voices separation (MSVS). While related to speech separation, MSVS faces unique challenges: data scarcity and the highly correlated nature of singing voices mixture. To address these issues, we propose UNMIXX with three key components: (1) musically informed mixing strategy to construct highly correlated, music-like mixtures, (2) cross-source attention that drives representations of two singers apart via reverse attention, and (3) magnitude penalty loss penalizing erroneously assigned interfering energy. UNMIXX not only addresses data scarcity by simulating realistic training data, but also excels at separating highly correlated mixtures through cross-source interactions at both the architectural and loss levels. Our extensive experiments demonstrate that UNMIXX greatly enhances performance, with SDRi gains exceeding 2.2 dB over prior work.