Asymmetric Clean Segments-Guided Self-Supervised Learning for Robust Speaker Verification

TL;DR

This paper proposes a novel self-supervised learning framework for speaker verification that leverages clean and augmented speech segments, significantly improving robustness and outperforming existing methods on Voxceleb1.

Contribution

It introduces a new contrastive training approach using clean and augmented segments with weighted loss to enhance speaker verification robustness.

Findings

Achieves 19% improvement over traditional methods.

Surpasses many state-of-the-art techniques.

Effective use of clean and augmented data in contrastive learning.

Abstract

Contrastive self-supervised learning (CSL) for speaker verification (SV) has drawn increasing interest recently due to its ability to exploit unlabeled data. Performing data augmentation on raw waveforms, such as adding noise or reverberation, plays a pivotal role in achieving promising results in SV. Data augmentation, however, demands meticulous calibration to ensure intact speaker-specific information, which is difficult to achieve without speaker labels. To address this issue, we introduce a novel framework by incorporating clean and augmented segments into the contrastive training pipeline. The clean segments are repurposed to pair with noisy segments to form additional positive and negative pairs. Moreover, the contrastive loss is weighted to increase the difference between the clean and augmented embeddings of different speakers. Experimental results on Voxceleb1 suggest that the proposed framework can achieve a remarkable 19% improvement over the conventional methods, and it surpasses many existing state-of-the-art techniques.