Disentangling Voice and Content with Self-Supervision for Speaker Recognition

TL;DR

This paper introduces a self-supervised disentanglement framework for speaker recognition that separates speaker traits from speech content, improving accuracy without additional data or training.

Contribution

It proposes a novel disentanglement approach using Gaussian inference layers and a self-supervision method that does not require extra labels beyond speaker identities.

Findings

Achieved 9.56% reduction in EER on VoxCeleb

Achieved 8.24% reduction in minDCF on SITW

Framework is practical and easily applicable without extra data or training

Abstract

For speaker recognition, it is difficult to extract an accurate speaker representation from speech because of its mixture of speaker traits and content. This paper proposes a disentanglement framework that simultaneously models speaker traits and content variability in speech. It is realized with the use of three Gaussian inference layers, each consisting of a learnable transition model that extracts distinct speech components. Notably, a strengthened transition model is specifically designed to model complex speech dynamics. We also propose a self-supervision method to dynamically disentangle content without the use of labels other than speaker identities. The efficacy of the proposed framework is validated via experiments conducted on the VoxCeleb and SITW datasets with 9.56% and 8.24% average reductions in EER and minDCF, respectively. Since neither additional model training nor data is specifically needed, it is easily applicable in practical use.