Why Did the x-Vector System Miss a Target Speaker? Impact of Acoustic Mismatch Upon Target Score on VoxCeleb Data

TL;DR

This paper investigates how acoustic mismatches between enrollment and test utterances affect the performance of x-vector speaker verification systems, revealing key mismatch factors and their impact on false rejections.

Contribution

It models the dependency of ASV scores on acoustic mismatch factors using a linear mixed effects model, providing interpretability and insights into system robustness.

Findings

F0 mean mismatch significantly affects target detection.

Formant frequency mismatches also influence false rejections.

X-vector systems show limited robustness to intra-speaker acoustic variations.

Abstract

Modern automatic speaker verification (ASV) relies heavily on machine learning implemented through deep neural networks. It can be difficult to interpret the output of these black boxes. In line with interpretative machine learning, we model the dependency of ASV detection score upon acoustic mismatch of the enrollment and test utterances. We aim to identify mismatch factors that explain target speaker misses (false rejections). We use distance in the first- and second-order statistics of selected acoustic features as the predictors in a linear mixed effects model, while a standard Kaldi x-vector system forms our ASV black-box. Our results on the VoxCeleb data reveal the most prominent mismatch factor to be in F0 mean, followed by mismatches associated with formant frequencies. Our findings indicate that x-vector systems lack robustness to intra-speaker variations.