Dr-Vectors: Decision Residual Networks and an Improved Loss for Speaker Recognition

TL;DR

This paper introduces Decision Residual Networks and an improved loss function for speaker recognition, enhancing the modeling of uncertainty and non-linear relationships, leading to significant performance improvements.

Contribution

It proposes a novel decision residual network architecture and a modified loss function to better capture uncertainty and improve speaker recognition accuracy.

Findings

Significant performance gains with the proposed methods.

Effective modeling of utterance-specific uncertainty.

Enhanced separation of same/different speaker scores.

Abstract

Many neural network speaker recognition systems model each speaker using a fixed-dimensional embedding vector. These embeddings are generally compared using either linear or 2nd-order scoring and, until recently, do not handle utterance-specific uncertainty. In this work we propose scoring these representations in a way that can capture uncertainty, enroll/test asymmetry and additional non-linear information. This is achieved by incorporating a 2nd-stage neural network (known as a decision network) as part of an end-to-end training regimen. In particular, we propose the concept of decision residual networks which involves the use of a compact decision network to leverage cosine scores and to model the residual signal that's needed. Additionally, we present a modification to the generalized end-to-end softmax loss function to target the separation of same/different speaker scores. We observed significant performance gains for the two techniques.