Speaker Contrastive Learning for Source Speaker Tracing

TL;DR

This paper introduces a speaker contrastive learning method for source speaker tracing in voice conversion scenarios, significantly improving the identification of source speakers in manipulated speech and enhancing speaker verification security.

Contribution

It proposes a novel contrastive learning approach that effectively captures source speaker information in converted speech, outperforming existing methods in the challenge.

Findings

Achieved the lowest EER of 16.788% on the challenge test set.

Secured first place in the IEEE SLT2024 Source Speaker Tracing Challenge.

Demonstrated the effectiveness of contrastive loss in source speaker identification.

Abstract

As a form of biometric authentication technology, the security of speaker verification systems is of utmost importance. However, SV systems are inherently vulnerable to various types of attacks that can compromise their accuracy and reliability. One such attack is voice conversion, which modifies a persons speech to sound like another person by altering various vocal characteristics. This poses a significant threat to SV systems. To address this challenge, the Source Speaker Tracing Challenge in IEEE SLT2024 aims to identify the source speaker information in manipulated speech signals. Specifically, SSTC focuses on source speaker verification against voice conversion to determine whether two converted speech samples originate from the same source speaker. In this study, we propose a speaker contrastive learning-based approach for source speaker tracing to learn the latent source speaker information in converted speech. To learn a more source-speaker-related representation, we employ speaker contrastive loss during the training of the embedding extractor. This speaker contrastive loss helps identify the true source speaker embedding among several distractor speaker embeddings, enabling the embedding extractor to learn the potentially possessing source speaker information present in the converted speech. Experiments demonstrate that our proposed speaker contrastive learning system achieves the lowest EER of 16.788% on the challenge test set, securing first place in the challenge.