Investigation of Different Calibration Methods for Deep Speaker Embedding based Verification Systems

TL;DR

This paper investigates various score calibration methods for deep speaker verification systems, emphasizing the importance of calibration in unknown conditions and evaluating classical, neural, and normalization approaches.

Contribution

It compares traditional and neural calibration methods, including novel neural approaches, and assesses the impact of score normalization on calibration performance.

Findings

In-domain data yields good calibration results.

Adaptive s-norm stabilizes score distributions.

Neural calibration methods have limitations across datasets.

Abstract

Deep speaker embedding extractors have already become new state-of-the-art systems in the speaker verification field. However, the problem of verification score calibration for such systems often remains out of focus. An irrelevant score calibration leads to serious issues, especially in the case of unknown acoustic conditions, even if we use a strong speaker verification system in terms of threshold-free metrics. This paper presents an investigation over several methods of score calibration: a classical approach based on the logistic regression model; the recently presented magnitude estimation network MagnetO that uses activations from the pooling layer of the trained deep speaker extractor and generalization of such approach based on separate scale and offset prediction neural networks. An additional focus of this research is to estimate the impact of score normalization on the calibration performance of the system. The obtained results demonstrate that there are no serious problems if in-domain development data are used for calibration tuning. Otherwise, a trade-off between good calibration performance and threshold-free system quality arises. In most cases using adaptive s-norm helps to stabilize score distributions and to improve system performance. Meanwhile, some experiments demonstrate that novel approaches have their limits in score stabilization on several datasets.