EDITnet: A Lightweight Network for Unsupervised Domain Adaptation in Speaker Verification

TL;DR

This paper introduces EDITnet, a lightweight unsupervised domain transfer network that improves speaker verification across languages by transferring embeddings into a common domain without requiring speaker labels.

Contribution

The paper presents a novel domain transfer network using a conditional variational auto-encoder and self-supervised learning, enabling efficient language mismatch mitigation without fine-tuning the embedding extractor.

Findings

EDITnet improves speaker verification accuracy by around 30% on Voxceleb and CN-Celeb datasets.

The method works with various embedding models like ECAPA-TDNN, TDNN, and SE-ResNet34.

Training is efficient and low-cost due to fixed embedding extraction models.

Abstract

Performance degradation caused by language mismatch is a common problem when applying a speaker verification system on speech data in different languages. This paper proposes a domain transfer network, named EDITnet, to alleviate the language-mismatch problem on speaker embeddings without requiring speaker labels. The network leverages a conditional variational auto-encoder to transfer embeddings from the target domain into the source domain. A self-supervised learning strategy is imposed on the transferred embeddings so as to increase the cosine distance between embeddings from different speakers. In the training process of the EDITnet, the embedding extraction model is fixed without fine-tuning, which renders the training efficient and low-cost. Experiments on Voxceleb and CN-Celeb show that the embeddings transferred by EDITnet outperform the un-transferred ones by around 30% with the ECAPA-TDNN512. Performance improvement can also be achieved with other embedding extraction models, e.g., TDNN, SE-ResNet34.