Advancing the dimensionality reduction of speaker embeddings for speaker diarisation: disentangling noise and informing speech activity

TL;DR

This paper introduces a new framework for reducing the dimensionality of speaker embeddings that effectively disentangles noise from speaker information, improving diarisation accuracy without system fusion.

Contribution

It proposes a novel disentanglement framework and utilizes speech activity vectors to enhance noise robustness in speaker embeddings.

Findings

Achieves state-of-the-art performance on four datasets

Effectively separates noise from speaker information

Improves diarisation accuracy without system fusion

Abstract

The objective of this work is to train noise-robust speaker embeddings adapted for speaker diarisation. Speaker embeddings play a crucial role in the performance of diarisation systems, but they often capture spurious information such as noise, adversely affecting performance. Our previous work has proposed an auto-encoder-based dimensionality reduction module to help remove the redundant information. However, they do not explicitly separate such information and have also been found to be sensitive to hyper-parameter values. To this end, we propose two contributions to overcome these issues: (i) a novel dimensionality reduction framework that can disentangle spurious information from the speaker embeddings; (ii) the use of speech activity vector to prevent the speaker code from representing the background noise. Through a range of experiments conducted on four datasets, our approach consistently demonstrates the state-of-the-art performance among models without system fusion.