Unsupervised Improved MVDR Beamforming for Sound Enhancement

TL;DR

This paper introduces UIMVDR, an unsupervised multi-channel sound separation method that leverages in-the-wild single-channel data, outperforming supervised models especially with limited training data.

Contribution

The paper presents UIMVDR, a novel unsupervised beamforming approach that enables multi-channel sound separation using only single-channel data, reducing data collection efforts.

Findings

UIMVDR outperforms supervised models in separation quality.

It generalizes well across different sound scenarios.

It reduces the need for large labeled datasets.

Abstract

Neural networks have recently become the dominant approach to sound separation. Their good performance relies on large datasets of isolated recordings. For speech and music, isolated single channel data are readily available; however the same does not hold in the multi-channel case, and with most other sound classes. Multi-channel methods have the potential to outperform single channel approaches as they can exploit both spatial and spectral features, but the lack of training data remains a challenge. We propose unsupervised improved minimum variation distortionless response (UIMVDR), which enables multi-channel separation to leverage in-the-wild single-channel data through unsupervised training and beamforming. Results show that UIMVDR generalizes well and improves separation performance compared to supervised models, particularly in cases with limited supervised data. By using data available online, it also reduces the effort required to gather data for multi-channel approaches.