Towards Low-distortion Multi-channel Speech Enhancement: The ESPNet-SE Submission to The L3DAS22 Challenge

TL;DR

This paper presents a novel multi-channel speech enhancement system combining DNN-based spectral mapping with linear beamforming, achieving top performance in the L3DAS22 challenge.

Contribution

It introduces a dual-DNN and linear beamformer architecture with iterative refinement for low-distortion multi-channel speech enhancement.

Findings

Ranked first in the L3DAS22 challenge with a metric of 0.984.

Outperformed the challenge baseline significantly.

Demonstrated effective integration of DNNs and beamforming for speech enhancement.

Abstract

This paper describes our submission to the L3DAS22 Challenge Task 1, which consists of speech enhancement with 3D Ambisonic microphones. The core of our approach combines Deep Neural Network (DNN) driven complex spectral mapping with linear beamformers such as the multi-frame multi-channel Wiener filter. Our proposed system has two DNNs and a linear beamformer in between. Both DNNs are trained to perform complex spectral mapping, using a combination of waveform and magnitude spectrum losses. The estimated signal from the first DNN is used to drive a linear beamformer, and the beamforming result, together with this enhanced signal, are used as extra inputs for the second DNN which refines the estimation. Then, from this new estimated signal, the linear beamformer and second DNN are run iteratively. The proposed method was ranked first in the challenge, achieving, on the evaluation set, a ranking metric of 0.984, versus 0.833 of the challenge baseline.