Mitigating Non-Target Speaker Bias in Guided Speaker Embedding

TL;DR

This paper addresses the issue of non-target speaker bias in guided speaker embeddings by proposing a method that leverages target speaker activity clues to improve performance in overlapping speech scenarios.

Contribution

It introduces an extension to global-statistics modules that incorporates target speaker activity, reducing bias and enhancing embedding quality.

Findings

Improved speaker verification accuracy across various overlap ratios.

Enhanced diarization performance on multiple datasets.

Reduction in bias caused by non-target speaker intervals.

Abstract

Obtaining high-quality speaker embeddings in multi-speaker conditions is crucial for many applications. A recently proposed guided speaker embedding framework, which utilizes speech activities of target and non-target speakers as clues, drastically improved embeddings under severe overlap with small degradation in low-overlap cases. However, since extreme overlaps are rare in natural conversations, this degradation cannot be overlooked. This paper first reveals that the degradation is caused by the global-statistics-based modules, widely used in speaker embedding extractors, being overly sensitive to intervals containing only non-target speakers. As a countermeasure, we propose an extension of such modules that exploit the target speaker activity clues, to compute statistics from intervals where the target is active. The proposed method improves speaker verification performance in both low and high overlap ratios, and diarization performance on multiple datasets.