EDSep: An Effective Diffusion-Based Method for Speech Source Separation

TL;DR

EDSep introduces a diffusion-based speech separation method that improves training and sampling efficiency, achieving superior results on multiple datasets compared to existing models.

Contribution

The paper presents EDSep, a novel diffusion-based approach with a new denoiser and stochastic sampler, enhancing speech separation performance and efficiency.

Findings

Outperforms existing diffusion models in speech separation

Achieves better separation quality on WSJ0-2mix, LRS2-2mix, VoxCeleb2-2mix

Demonstrates improved training and sampling efficiency

Abstract

Generative models have attracted considerable attention for speech separation tasks, and among these, diffusion-based methods are being explored. Despite the notable success of diffusion techniques in generation tasks, their adaptation to speech separation has encountered challenges, notably slow convergence and suboptimal separation outcomes. To address these issues and enhance the efficacy of diffusion-based speech separation, we introduce EDSep, a novel single-channel method grounded in score matching via stochastic differential equation (SDE). This method enhances generative modeling for speech source separation by optimizing training and sampling efficiency. Specifically, a novel denoiser function is proposed to approximate data distributions, which obtains ideal denoiser outputs. Additionally, a stochastic sampler is carefully designed to resolve the reverse SDE during the sampling process, gradually separating speech from mixtures. Extensive experiments on databases such as WSJ0-2mix, LRS2-2mix, and VoxCeleb2-2mix demonstrate our proposed method's superior performance over existing diffusion and discriminative models, validating its efficacy.