Masked Proxy Loss For Text-Independent Speaker Verification

TL;DR

This paper introduces Masked Proxy loss and Multinomial Masked Proxy loss for text-independent speaker verification, combining proxy and pair-based relationships to improve performance and achieve state-of-the-art results.

Contribution

It proposes novel Masked Proxy and Multinomial Masked Proxy losses that enhance metric learning for speaker verification by leveraging both proxy and pair relationships.

Findings

Achieved state-of-the-art EER on VoxCeleb test set.

Effectively combines proxy-based and pair-based relationships.

Improves robustness over existing metric learning methods.

Abstract

Open-set speaker recognition can be regarded as a metric learning problem, which is to maximize inter-class variance and minimize intra-class variance. Supervised metric learning can be categorized into entity-based learning and proxy-based learning. Most of the existing metric learning objectives like Contrastive, Triplet, Prototypical, GE2E, etc all belong to the former division, the performance of which is either highly dependent on sample mining strategy or restricted by insufficient label information in the mini-batch. Proxy-based losses mitigate both shortcomings, however, fine-grained connections among entities are either not or indirectly leveraged. This paper proposes a Masked Proxy (MP) loss which directly incorporates both proxy-based relationships and pair-based relationships. We further propose Multinomial Masked Proxy (MMP) loss to leverage the hardness of speaker pairs. These methods have been applied to evaluate on VoxCeleb test set and reach state-of-the-art Equal Error Rate(EER).