Multi-scale speaker embedding-based graph attention networks for speaker diarisation

TL;DR

This paper introduces a novel graph attention network approach for multi-scale speaker diarisation, effectively leveraging scale information and affinity matrices to improve speaker recognition accuracy across diverse datasets.

Contribution

It proposes a new graph attention network method with scale indicators and affinity matrix utilization for enhanced multi-scale speaker diarisation.

Findings

Speaker confusion reduced by over 10% on average

Effective utilization of multi-scale embeddings

Improved accuracy across various datasets

Abstract

The objective of this work is effective speaker diarisation using multi-scale speaker embeddings. Typically, there is a trade-off between the ability to recognise short speaker segments and the discriminative power of the embedding, according to the segment length used for embedding extraction. To this end, recent works have proposed the use of multi-scale embeddings where segments with varying lengths are used. However, the scores are combined using a weighted summation scheme where the weights are fixed after the training phase, whereas the importance of segment lengths can differ with in a single session. To address this issue, we present three key contributions in this paper: (1) we propose graph attention networks for multi-scale speaker diarisation; (2) we design scale indicators to utilise scale information of each embedding; (3) we adapt the attention-based aggregation to utilise a pre-computed affinity matrix from multi-scale embeddings. We demonstrate the effectiveness of our method in various datasets where the speaker confusion which constitutes the primary metric drops over 10% in average relative compared to the baseline.