ParaNoise-SV: Integrated Approach for Noise-Robust Speaker Verification with Parallel Joint Learning of Speech Enhancement and Noise Extraction

TL;DR

ParaNoise-SV introduces a dual U-Net architecture that explicitly models noise and enhances speech, significantly improving noise-robust speaker verification performance by reducing error rates.

Contribution

It presents a novel parallel joint learning framework with explicit noise modeling, advancing noise robustness in speaker verification.

Findings

Achieves 8.4% lower EER than previous models

Effectively separates noise from speech during training

Enhances speaker verification accuracy in noisy environments

Abstract

Noise-robust speaker verification leverages joint learning of speech enhancement (SE) and speaker verification (SV) to improve robustness. However, prevailing approaches rely on implicit noise suppression, which struggles to separate noise from speaker characteristics as they do not explicitly distinguish noise from speech during training. Although integrating SE and SV helps, it remains limited in handling noise effectively. Meanwhile, recent SE studies suggest that explicitly modeling noise, rather than merely suppressing it, enhances noise resilience. Reflecting this, we propose ParaNoise-SV, with dual U-Nets combining a noise extraction (NE) network and a speech enhancement (SE) network. The NE U-Net explicitly models noise, while the SE U-Net refines speech with guidance from NE through parallel connections, preserving speaker-relevant features. Experimental results show that ParaNoise-SV achieves a relatively 8.4% lower equal error rate (EER) than previous joint SE-SV models.