DESNet: A Multi-channel Network for Simultaneous Speech Dereverberation, Enhancement and Separation

TL;DR

This paper introduces DESNet, a multi-channel neural network that simultaneously performs speech dereverberation, enhancement, and separation, utilizing attentional feature selection, a deep complex RNN, and a cascaded WPE for improved speech processing.

Contribution

The paper presents a novel multi-channel network with attentional feature selection and a staged training strategy for joint speech dereverberation, enhancement, and separation.

Findings

Outperforms DCCRN and state-of-the-art structures in non-dereverberated speech enhancement.

Shows improved performance over cascaded WPE-DCCRN in dereverberated scenarios.

Effective in simultaneous dereverberation, enhancement, and separation tasks.

Abstract

In this paper, we propose a multi-channel network for simultaneous speech dereverberation, enhancement and separation (DESNet). To enable gradient propagation and joint optimization, we adopt the attentional selection mechanism of the multi-channel features, which is originally proposed in end-to-end unmixing, fixed-beamforming and extraction (E2E-UFE) structure. Furthermore, the novel deep complex convolutional recurrent network (DCCRN) is used as the structure of the speech unmixing and the neural network based weighted prediction error (WPE) is cascaded beforehand for speech dereverberation. We also introduce the staged SNR strategy and symphonic loss for the training of the network to further improve the final performance. Experiments show that in non-dereverberated case, the proposed DESNet outperforms DCCRN and most state-of-the-art structures in speech enhancement and separation, while in dereverberated scenario, DESNet also shows improvements over the cascaded WPE-DCCRN networks.