PlumberNet: Fixing interference leakage after GEV beamforming

TL;DR

This paper introduces PlumberNet, a method that improves speech enhancement by accurately estimating spatial covariance matrices from DoA and pairwise masks, reducing interference leakage after GEV beamforming.

Contribution

The work presents a novel approach to estimate spatial covariance matrices from DoA and pairwise masks, enhancing postfiltering performance after GEV beamforming.

Findings

Improved speech quality with reduced interference leakage.

Accurate SCM estimation from DoA and masks.

Enhanced postfiltering performance.

Abstract

Spatial filters can exploit deep-learning-based speech enhancement models to increase their reliability in scenarios with multiple speech sources scenarios. To further improve speech quality, it is common to perform postfiltering on the estimated target speech obtained with spatial filtering. In this work, Generalized Eigenvalue (GEV) beamforming is employed to provide the leakage estimation, along with the estimation of the target speech, to be later used for postfiltering. This improves the enhancement performance over a postfilter that uses the target speech and a reference microphone signal. This work also demonstrates that the spatial covariance matrices (SCMs) can be accurately estimated from the direction of arrival (DoA) of the target and a discriminative selection amongst the pairwise estimated time-frequency masks.