A Modulation-Domain Loss for Neural-Network-based Real-time Speech Enhancement

TL;DR

This paper introduces a novel modulation-domain loss function for neural-network-based real-time speech enhancement, improving speech quality and intelligibility predictions without extra inference cost.

Contribution

It proposes a modulation-domain loss derived from learnable spectro-temporal receptive fields optimized for speech enhancement tasks.

Findings

Improved objective speech quality metrics

Enhanced speech intelligibility predictions

No additional inference computation required

Abstract

We describe a modulation-domain loss function for deep-learning-based speech enhancement systems. Learnable spectro-temporal receptive fields (STRFs) were adapted to optimize for a speaker identification task. The learned STRFs were then used to calculate a weighted mean-squared error (MSE) in the modulation domain for training a speech enhancement system. Experiments showed that adding the modulation-domain MSE to the MSE in the spectro-temporal domain substantially improved the objective prediction of speech quality and intelligibility for real-time speech enhancement systems without incurring additional computation during inference.