Graph-based Multi-View Fusion and Local Adaptation: Mitigating Within-Household Confusability for Speaker Identification

TL;DR

This paper introduces a graph-based semi-supervised learning method that enhances household speaker identification accuracy by leveraging multi-view graphs and local adaptation, effectively addressing confusability and demographic biases.

Contribution

It presents a novel graph normalization and multi-signal fusion technique for household SID that does not require embedding tuning or additional training, improving robustness and fairness.

Findings

Consistent performance improvements across diverse households.

Effective mitigation of confusability issues.

Robustness to demographic imbalances.

Abstract

Speaker identification (SID) in the household scenario (e.g., for smart speakers) is an important but challenging problem due to limited number of labeled (enrollment) utterances, confusable voices, and demographic imbalances. Conventional speaker recognition systems generalize from a large random sample of speakers, causing the recognition to underperform for households drawn from specific cohorts or otherwise exhibiting high confusability. In this work, we propose a graph-based semi-supervised learning approach to improve household-level SID accuracy and robustness with locally adapted graph normalization and multi-signal fusion with multi-view graphs. Unlike other work on household SID, fairness, and signal fusion, this work focuses on speaker label inference (scoring) and provides a simple solution to realize household-specific adaptation and multi-signal fusion without tuning the embeddings or training a fusion network. Experiments on the VoxCeleb dataset demonstrate that our approach consistently improves the performance across households with different customer cohorts and degrees of confusability.