Learning from human perception to improve automatic speaker verification in style-mismatched conditions

TL;DR

This paper introduces a novel training loss function inspired by human perception to enhance automatic speaker verification performance under style-mismatched conditions, demonstrating significant improvements across multiple datasets.

Contribution

The paper proposes the CllrCE loss, integrating human perceptual insights into training to better handle style variability in speaker verification systems.

Findings

CllrCE loss improves EER by up to 66% on UCLA database.

Significant reductions in minDCF observed with the new loss function.

Performance gains are consistent with conditioning in SITW evaluations.

Abstract

Our prior experiments show that humans and machines seem to employ different approaches to speaker discrimination, especially in the presence of speaking style variability. The experiments examined read versus conversational speech. Listeners focused on speaker-specific idiosyncrasies while "telling speakers together", and on relative distances in a shared acoustic space when "telling speakers apart". However, automatic speaker verification (ASV) systems use the same loss function irrespective of target or non-target trials. To improve ASV performance in the presence of style variability, insights learnt from human perception are used to design a new training loss function that we refer to as "CllrCE loss". CllrCE loss uses both speaker-specific idiosyncrasies and relative acoustic distances between speakers to train the ASV system. When using the UCLA speaker variability database, in the x-vector and conditioning setups, CllrCE loss results in significant relative improvements in EER by 1-66%, and minDCF by 1-31% and 1-56%, respectively, when compared to the x-vector baseline. Using the SITW evaluation tasks, which involve different conversational speech tasks, the proposed loss combined with self-attention conditioning results in significant relative improvements in EER by 2-5% and minDCF by 6-12% over baseline. In the SITW case, performance improvements were consistent only with conditioning.