Speaker diarization with session-level speaker embedding refinement using graph neural networks

TL;DR

This paper introduces a novel graph neural network approach to refine speaker embeddings locally within sessions, significantly improving diarization accuracy and achieving state-of-the-art results on benchmark datasets.

Contribution

First application of GNNs for session-level speaker embedding refinement, enhancing speaker separation in diarization systems.

Findings

Refined embeddings outperform original embeddings in clustering accuracy.

System achieves state-of-the-art results on NIST SRE 2000 CALLHOME.

Significant improvements on both simulated and real meeting data.

Abstract

Deep speaker embedding models have been commonly used as a building block for speaker diarization systems; however, the speaker embedding model is usually trained according to a global loss defined on the training data, which could be sub-optimal for distinguishing speakers locally in a specific meeting session. In this work we present the first use of graph neural networks (GNNs) for the speaker diarization problem, utilizing a GNN to refine speaker embeddings locally using the structural information between speech segments inside each session. The speaker embeddings extracted by a pre-trained model are remapped into a new embedding space, in which the different speakers within a single session are better separated. The model is trained for linkage prediction in a supervised manner by minimizing the difference between the affinity matrix constructed by the refined embeddings and the ground-truth adjacency matrix. Spectral clustering is then applied on top of the refined embeddings. We show that the clustering performance of the refined speaker embeddings outperforms the original embeddings significantly on both simulated and real meeting data, and our system achieves the state-of-the-art result on the NIST SRE 2000 CALLHOME database.