WPD++: An Improved Neural Beamformer for Simultaneous Speech Separation and Dereverberation

TL;DR

This paper introduces WPD++, an improved neural beamformer that enhances speech separation and dereverberation by utilizing spatio-temporal correlation and a multi-objective loss, leading to better speech quality and ASR performance.

Contribution

WPD++ advances neural beamforming by integrating spatio-temporal correlation and a multi-objective loss for joint training, improving speech separation and dereverberation.

Findings

WPD++ outperforms state-of-the-art beamformers in speech quality.

WPD++ reduces word error rate in ASR tasks.

Enhanced speech signals show significant quality improvements.

Abstract

This paper aims at eliminating the interfering speakers' speech, additive noise, and reverberation from the noisy multi-talker speech mixture that benefits automatic speech recognition (ASR) backend. While the recently proposed Weighted Power minimization Distortionless response (WPD) beamformer can perform separation and dereverberation simultaneously, the noise cancellation component still has the potential to progress. We propose an improved neural WPD beamformer called "WPD++" by an enhanced beamforming module in the conventional WPD and a multi-objective loss function for the joint training. The beamforming module is improved by utilizing the spatio-temporal correlation. A multi-objective loss, including the complex spectra domain scale-invariant signal-to-noise ratio (C-Si-SNR) and the magnitude domain mean square error (Mag-MSE), is properly designed to make multiple constraints on the enhanced speech and the desired power of the dry clean signal. Joint training is conducted to optimize the complex-valued mask estimator and the WPD++ beamformer in an end-to-end way. The results show that the proposed WPD++ outperforms several state-of-the-art beamformers on the enhanced speech quality and word error rate (WER) of ASR.