PL-EESR: Perceptual Loss Based END-TO-END Robust Speaker Representation Extraction

TL;DR

PL-EESR is an end-to-end deep learning framework that enhances speaker representation robustness by balancing noise suppression with perceptual quality, improving speaker verification in noisy and clean environments.

Contribution

It introduces a perceptual loss-based optimization for robust speaker embedding extraction, addressing noise suppression and speech distortion issues simultaneously.

Findings

Outperforms baseline in noisy environments

Maintains speaker information without distortion

Effective in both noisy and clean conditions

Abstract

Speech enhancement aims to improve the perceptual quality of the speech signal by suppression of the background noise. However, excessive suppression may lead to speech distortion and speaker information loss, which degrades the performance of speaker embedding extraction. To alleviate this problem, we propose an end-to-end deep learning framework, dubbed PL-EESR, for robust speaker representation extraction. This framework is optimized based on the feedback of the speaker identification task and the high-level perceptual deviation between the raw speech signal and its noisy version. We conducted speaker verification tasks in both noisy and clean environment respectively to evaluate our system. Compared to the baseline, our method shows better performance in both clean and noisy environments, which means our method can not only enhance the speaker relative information but also avoid adding distortions.