Multi-task single channel speech enhancement using speech presence probability as a secondary task training target

TL;DR

This paper introduces a multi-task deep learning approach for single-channel speech enhancement that jointly estimates the Wiener gain and speech presence probability, resulting in improved dereverberation and noise reduction performance.

Contribution

It proposes a novel multi-task learning framework using speech presence probability as a secondary task to enhance speech enhancement robustness.

Findings

Multi-task learning improves speech enhancement performance.

Joint estimation of Wiener gain and SPP yields better dereverberation.

Adaptive loss weighting enhances training effectiveness.

Abstract

To cope with reverberation and noise in single channel acoustic scenarios, typical supervised deep neural network~(DNN)-based techniques learn a mapping from reverberant and noisy input features to a user-defined target. Commonly used targets are the desired signal magnitude, a time-frequency mask such as the Wiener gain, or the interference power spectral density and signal-to-interference ratio that can be used to compute a time-frequency mask. In this paper, we propose to incorporate multi-task learning in such DNN-based enhancement techniques by using speech presence probability (SPP) estimation as a secondary task assisting the target estimation in the main task. The advantage of multi-task learning lies in sharing domain-specific information between the two tasks (i.e., target and SPP estimation) and learning more generalizable and robust representations. To simultaneously learn both tasks, we propose to use the adaptive weighting method of losses derived from the homoscedastic uncertainty of tasks. Simulation results show that the dereverberation and noise reduction performance of a single-task DNN trained to directly estimate the Wiener gain is higher than the performance of single-task DNNs trained to estimate the desired signal magnitude, the interference power spectral density, or the signal-to-interference ratio. Incorporating the proposed multi-task learning scheme to jointly estimate the Wiener gain and the SPP increases the dereverberation and noise reduction further.