Convolutive Prediction for Reverberant Speech Separation

TL;DR

This paper introduces a convolutive prediction approach combined with deep neural networks and beamforming to improve speech separation and dereverberation in reverberant environments, achieving state-of-the-art results.

Contribution

It proposes a novel convolutive prediction method for dereverberation, integrating DNNs and beamforming for enhanced speaker separation in reverberant conditions.

Findings

Achieved state-of-the-art separation results on SMS-WSJ corpus.

Effectively identified and removed reverberant copies of speech signals.

Enhanced separation performance using combined convolutive prediction and beamforming.

Abstract

We investigate the effectiveness of convolutive prediction, a novel formulation of linear prediction for speech dereverberation, for speaker separation in reverberant conditions. The key idea is to first use a deep neural network (DNN) to estimate the direct-path signal of each speaker, and then identify delayed and decayed copies of the estimated direct-path signal. Such copies are likely due to reverberation, and can be directly removed for dereverberation or used as extra features for another DNN to perform better dereverberation and separation. To identify such copies, we solve a linear regression problem per frequency efficiently in the time-frequency (T-F) domain to estimate the underlying room impulse response (RIR). In the multi-channel extension, we perform minimum variance distortionless response (MVDR) beamforming on the outputs of convolutive prediction. The beamforming and dereverberation results are used as extra features for a second DNN to perform better separation and dereverberation. State-of-the-art results are obtained on the SMS-WSJ corpus.