Prototype and Instance Contrastive Learning for Unsupervised Domain Adaptation in Speaker Verification

TL;DR

This paper introduces PICL, a dual-level contrastive learning approach for unsupervised domain adaptation in speaker verification, improving robustness and generalization across diverse mismatch scenarios.

Contribution

The paper proposes a novel dual-level contrastive learning method, combining prototype and instance contrastive learning, for better unsupervised domain adaptation in speaker verification.

Findings

Achieved state-of-the-art performance on multiple datasets.

Demonstrated improved robustness across various mismatch scenarios.

Validated the generalization capability of PICL.

Abstract

Speaker verification system trained on one domain usually suffers performance degradation when applied to another domain. To address this challenge, researchers commonly use feature distribution matching-based methods in unsupervised domain adaptation scenarios where some unlabeled target domain data is available. However, these methods often have limited performance improvement and lack generalization in various mismatch situations. In this paper, we propose Prototype and Instance Contrastive Learning (PICL), a novel method for unsupervised domain adaptation in speaker verification through dual-level contrastive learning. For prototype contrastive learning, we generate pseudo labels via clustering to create dynamically updated prototype representations, aligning instances with their corresponding class or cluster prototypes. For instance contrastive learning, we minimize the distance between different views or augmentations of the same instance, ensuring robust and invariant representations resilient to variations like noise. This dual-level approach provides both high-level and low-level supervision, leading to improved generalization and robustness of the speaker verification model. Unlike previous studies that only evaluated mismatches in one situation, we have conducted relevant explorations on various datasets and achieved state-of-the-art performance currently, which also proves the generalization of our method.