ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior

TL;DR

ArrayDPS introduces an unsupervised, array-agnostic speech separation method using diffusion priors, effectively separating sources without prior microphone array information and outperforming baseline methods.

Contribution

The paper presents ArrayDPS, a novel unsupervised diffusion-based approach for blind speech separation that does not require microphone array details.

Findings

Outperforms baseline unsupervised methods in SDR

Comparable to supervised methods in separation quality

Operates without microphone array information

Abstract

Blind Speech Separation (BSS) aims to separate multiple speech sources from audio mixtures recorded by a microphone array. The problem is challenging because it is a blind inverse problem, i.e., the microphone array geometry, the room impulse response (RIR), and the speech sources, are all unknown. We propose ArrayDPS to solve the BSS problem in an unsupervised, array-agnostic, and generative manner. The core idea builds on diffusion posterior sampling (DPS), but unlike DPS where the likelihood is tractable, ArrayDPS must approximate the likelihood by formulating a separate optimization problem. The solution to the optimization approximates room acoustics and the relative transfer functions between microphones. These approximations, along with the diffusion priors, iterate through the ArrayDPS sampling process and ultimately yield separated voice sources. We only need a simple single-speaker speech diffusion model as a prior along with the mixtures recorded at the microphones; no microphone array information is necessary. Evaluation results show that ArrayDPS outperforms all baseline unsupervised methods while being comparable to supervised methods in terms of SDR. Audio demos are provided at: https://arraydps.github.io/ArrayDPSDemo/.