Multi-Objective Learning and Mask-Based Post-Processing for Deep Neural Network Based Speech Enhancement

TL;DR

This paper introduces a multi-objective deep learning framework for speech enhancement that jointly learns primary and secondary targets, improving performance and enabling effective post-processing techniques.

Contribution

It presents a novel joint learning architecture for primary and secondary speech features, enhancing enhancement quality and enabling mask-based post-processing.

Findings

Joint LPS and MFCC learning improves speech enhancement.

IBM-based post-processing further enhances speech quality.

The framework outperforms traditional single-target methods.

Abstract

We propose a multi-objective framework to learn both secondary targets not directly related to the intended task of speech enhancement (SE) and the primary target of the clean log-power spectra (LPS) features to be used directly for constructing the enhanced speech signals. In deep neural network (DNN) based SE we introduce an auxiliary structure to learn secondary continuous features, such as mel-frequency cepstral coefficients (MFCCs), and categorical information, such as the ideal binary mask (IBM), and integrate it into the original DNN architecture for joint optimization of all the parameters. This joint estimation scheme imposes additional constraints not available in the direct prediction of LPS, and potentially improves the learning of the primary target. Furthermore, the learned secondary information as a byproduct can be used for other purposes, e.g., the IBM-based post-processing in this work. A series of experiments show that joint LPS and MFCC learning improves the SE performance, and IBM-based post-processing further enhances listening quality of the reconstructed speech.