Monaural Speech Enhancement using Deep Neural Networks by Maximizing a Short-Time Objective Intelligibility Measure

TL;DR

This paper introduces a DNN-based speech enhancement system that directly maximizes an approximation of the STOI measure, leading to significant improvements in speech intelligibility across various noise conditions.

Contribution

It formalizes an approximate-STOI cost function for DNN training and demonstrates its effectiveness compared to traditional MSE-based and spectral amplitude methods.

Findings

Achieves large improvements in speech intelligibility in simulations.

Performs comparably to MSE-based systems in intelligibility.

Matches traditional spectral amplitude enhancement in intelligibility.

Abstract

In this paper we propose a Deep Neural Network (DNN) based Speech Enhancement (SE) system that is designed to maximize an approximation of the Short-Time Objective Intelligibility (STOI) measure. We formalize an approximate-STOI cost function and derive analytical expressions for the gradients required for DNN training and show that these gradients have desirable properties when used together with gradient based optimization techniques. We show through simulation experiments that the proposed SE system achieves large improvements in estimated speech intelligibility, when tested on matched and unmatched natural noise types, at multiple signal-to-noise ratios. Furthermore, we show that the SE system, when trained using an approximate-STOI cost function performs on par with a system trained with a mean square error cost applied to short-time temporal envelopes. Finally, we show that the proposed SE system performs on par with a traditional DNN based Short-Time Spectral Amplitude (STSA) SE system in terms of estimated speech intelligibility. These results are important because they suggest that traditional DNN based STSA SE systems might be optimal in terms of estimated speech intelligibility.