Speaker attribution with voice profiles by graph-based semi-supervised learning

TL;DR

This paper introduces a graph-based semi-supervised learning approach for speaker attribution in meetings, leveraging speaker embeddings and graph structures to significantly improve accuracy over traditional methods.

Contribution

It proposes a novel application of graph neural networks and label propagation for speaker attribution, utilizing structural information from speech segment graphs.

Findings

Reduced speaker attribution error by up to 68%

Effective utilization of speaker embeddings and graph structure

Improved performance over baseline methods

Abstract

Speaker attribution is required in many real-world applications, such as meeting transcription, where speaker identity is assigned to each utterance according to speaker voice profiles. In this paper, we propose to solve the speaker attribution problem by using graph-based semi-supervised learning methods. A graph of speech segments is built for each session, on which segments from voice profiles are represented by labeled nodes while segments from test utterances are unlabeled nodes. The weight of edges between nodes is evaluated by the similarities between the pretrained speaker embeddings of speech segments. Speaker attribution then becomes a semi-supervised learning problem on graphs, on which two graph-based methods are applied: label propagation (LP) and graph neural networks (GNNs). The proposed approaches are able to utilize the structural information of the graph to improve speaker attribution performance. Experimental results on real meeting data show that the graph based approaches reduce speaker attribution error by up to 68% compared to a baseline speaker identification approach that processes each utterance independently.