A sticky HDP-HMM with application to speaker diarization

TL;DR

This paper introduces an augmented sticky HDP-HMM model for speaker diarization, effectively handling unknown numbers of speakers and over-segmentation issues, achieving state-of-the-art results on benchmark data.

Contribution

It proposes a novel augmented sticky HDP-HMM with a scalable sampling algorithm for improved speaker diarization without prior speaker count knowledge.

Findings

Achieves state-of-the-art diarization accuracy on benchmark datasets.

Effectively controls over-segmentation and switching rates.

Handles nonparametric emission distributions.

Abstract

We consider the problem of speaker diarization, the problem of segmenting an audio recording of a meeting into temporal segments corresponding to individual speakers. The problem is rendered particularly difficult by the fact that we are not allowed to assume knowledge of the number of people participating in the meeting. To address this problem, we take a Bayesian nonparametric approach to speaker diarization that builds on the hierarchical Dirichlet process hidden Markov model (HDP-HMM) of Teh et al. [J. Amer. Statist. Assoc. 101 (2006) 1566--1581]. Although the basic HDP-HMM tends to over-segment the audio data---creating redundant states and rapidly switching among them---we describe an augmented HDP-HMM that provides effective control over the switching rate. We also show that this augmentation makes it possible to treat emission distributions nonparametrically. To scale the resulting architecture to realistic diarization problems, we develop a sampling algorithm that employs a truncated approximation of the Dirichlet process to jointly resample the full state sequence, greatly improving mixing rates. Working with a benchmark NIST data set, we show that our Bayesian nonparametric architecture yields state-of-the-art speaker diarization results.