Speaker-conversation factorial designs for diarization error analysis

TL;DR

This paper introduces a factorial design methodology to analyze how acoustic and conversational factors independently affect speaker diarization accuracy, helping to identify causes of errors and guide improvements.

Contribution

It proposes a novel factorial remixing approach to disentangle effects of acoustics and conversation structure on diarization errors, demonstrated through analysis of multiple systems.

Findings

Large accuracy disparities due to conversational structure in baseline system

Shorter subsegment systems show reduced structure-related errors

Method helps identify factors influencing diarization performance

Abstract

Speaker diarization accuracy can be affected by both acoustics and conversation characteristics. Determining the cause of diarization errors is difficult because speaker voice acoustics and conversation structure co-vary, and the interactions between acoustics, conversational structure, and diarization accuracy are complex. This paper proposes a methodology that can distinguish independent marginal effects of acoustic and conversation characteristics on diarization accuracy by remixing conversations in a factorial design. As an illustration, this approach is used to investigate gender-related and language-related accuracy differences with three diarization systems: a baseline system using subsegment x-vector clustering, a variant of it with shorter subsegments, and a third system based on a Bayesian hidden Markov model. Our analysis shows large accuracy disparities for the baseline system primarily due to conversational structure, which are partially mitigated in the other two systems. The illustration thus demonstrates how the methodology can be used to identify and guide diarization model improvements.