Joint Noise Reduction and Listening Enhancement for Full-End Speech Enhancement

TL;DR

This paper proposes a deep learning-based joint framework that combines noise reduction and listening enhancement to improve speech intelligibility in noisy environments, addressing both speaker and listener noise sources.

Contribution

It introduces a novel integrated approach that sequentially applies noise reduction and listening enhancement, outperforming traditional disjoint methods in speech clarity and intelligibility.

Findings

Significantly outperforms disjoint processing methods

Improves speech intelligibility in noisy environments

Achieves promising results on speech evaluation metrics

Abstract

Speech enhancement (SE) methods mainly focus on recovering clean speech from noisy input. In real-world speech communication, however, noises often exist in not only speaker but also listener environments. Although SE methods can suppress the noise contained in the speaker's voice, they cannot deal with the noise that is physically present in the listener side. To address such a complicated but common scenario, we investigate a deep learning-based joint framework integrating noise reduction (NR) with listening enhancement (LE), in which the NR module first suppresses noise and the LE module then modifies the denoised speech, i.e., the output of the NR module, to further improve speech intelligibility. The enhanced speech can thus be less noisy and more intelligible for listeners. Experimental results show that our proposed method achieves promising results and significantly outperforms the disjoint processing methods in terms of various speech evaluation metrics.