Clustering-based hard negative sampling for supervised contrastive speaker verification

TL;DR

This paper introduces CHNS, a clustering-based hard negative sampling method that enhances supervised contrastive learning for speaker verification, significantly improving performance over existing methods.

Contribution

The paper proposes a novel clustering-based hard negative sampling technique tailored for supervised contrastive speaker verification, outperforming existing approaches.

Findings

CHNS outperforms baseline supervised contrastive methods.

CHNS achieves up to 18% relative reduction in EER and minDCF.

Method is effective on lightweight models and the VoxCeleb dataset.

Abstract

In speaker verification, contrastive learning is gaining popularity as an alternative to the traditionally used classification-based approaches. Contrastive methods can benefit from an effective use of hard negative pairs, which are different-class samples particularly challenging for a verification model due to their similarity. In this paper, we propose CHNS - a clustering-based hard negative sampling method, dedicated for supervised contrastive speaker representation learning. Our approach clusters embeddings of similar speakers, and adjusts batch composition to obtain an optimal ratio of hard and easy negatives during contrastive loss calculation. Experimental evaluation shows that CHNS outperforms a baseline supervised contrastive approach with and without loss-based hard negative sampling, as well as a state-of-the-art classification-based approach to speaker verification by as much as 18 % relative EER and minDCF on the VoxCeleb dataset using two lightweight model architectures.