End-to-end losses based on speaker basis vectors and all-speaker hard negative mining for speaker verification

TL;DR

This paper introduces two novel end-to-end loss functions for speaker verification that leverage speaker basis vectors, enabling consideration of all speakers during training and improving inter-speaker variation and hard negative mining.

Contribution

The proposed loss functions utilize trainable speaker basis vectors to incorporate all speakers in training, enhancing speaker verification performance over traditional methods.

Findings

Improved speaker verification accuracy on VoxCeleb datasets.

Effective incorporation of all speakers in loss calculations.

Enhanced inter-speaker variation and hard negative mining capabilities.

Abstract

In recent years, speaker verification has primarily performed using deep neural networks that are trained to output embeddings from input features such as spectrograms or Mel-filterbank energies. Studies that design various loss functions, including metric learning have been widely explored. In this study, we propose two end-to-end loss functions for speaker verification using the concept of speaker bases, which are trainable parameters. One loss function is designed to further increase the inter-speaker variation, and the other is designed to conduct the identical concept with hard negative mining. Each speaker basis is designed to represent the corresponding speaker in the process of training deep neural networks. In contrast to the conventional loss functions that can consider only a limited number of speakers included in a mini-batch, the proposed loss functions can consider all the speakers in the training set regardless of the mini-batch composition. In particular, the proposed loss functions enable hard negative mining and calculations of between-speaker variations with consideration of all speakers. Through experiments on VoxCeleb1 and VoxCeleb2 datasets, we confirmed that the proposed loss functions could supplement conventional softmax and center loss functions.