Compositional embedding models for speaker identification and diarization with simultaneous speech from 2+ speakers

TL;DR

This paper introduces a compositional embedding approach for speaker diarization that effectively handles overlapping speech from multiple speakers, outperforming traditional methods and achieving state-of-the-art accuracy on real-world data.

Contribution

The paper presents a novel compositional embedding model that enables set-union operations in the embedding space for improved multi-speaker diarization.

Findings

Outperforms traditional speaker embedding methods on synthesized data

Achieves state-of-the-art diarization accuracy on AMI Headset Mix corpus

Demonstrates effectiveness in handling overlapping speech from multiple speakers

Abstract

We propose a new method for speaker diarization that can handle overlapping speech with 2+ people. Our method is based on compositional embeddings [1]: Like standard speaker embedding methods such as x-vector [2], compositional embedding models contain a function f that separates speech from different speakers. In addition, they include a composition function g to compute set-union operations in the embedding space so as to infer the set of speakers within the input audio. In an experiment on multi-person speaker identification using synthesized LibriSpeech data, the proposed method outperforms traditional embedding methods that are only trained to separate single speakers (not speaker sets). In a speaker diarization experiment on the AMI Headset Mix corpus, we achieve state-of-the-art accuracy (DER=22.93%), slightly higher than the previous best result (23.82% from [3]).