Improving Diarization Robustness using Diversification, Randomization and the DOVER Algorithm

TL;DR

This paper enhances speaker diarization robustness by combining multiple hypotheses through the DOVER algorithm and introducing diversification and randomization techniques to mitigate hyperparameter sensitivity.

Contribution

It proposes a novel approach to improve diarization robustness by averaging diverse clustering outputs using DOVER and incorporating pseudo-random merge choices.

Findings

More robust diarization results across datasets.

Improved overall diarization accuracy.

Effective mitigation of hyperparameter sensitivity.

Abstract

Speaker diarization based on bottom-up clustering of speech segments by acoustic similarity is often highly sensitive to the choice of hyperparameters, such as the initial number of clusters and feature weighting. Optimizing these hyperparameters is difficult and often not robust across different data sets. We recently proposed the DOVER algorithm for combining multiple diarization hypotheses by voting. Here we propose to mitigate the robustness problem in diarization by using DOVER to average across different parameter choices. We also investigate the combination of diverse outputs obtained by following different merge choices pseudo-randomly in the course of clustering, thereby mitigating the greediness of best-first clustering. We show on two conference meeting data sets drawn from NIST evaluations that the proposed methods indeed yield more robust, and in several cases overall improved, results.