Simultaneous Diarization and Separation of Meetings through the Integration of Statistical Mixture Models

TL;DR

This paper introduces a joint statistical framework combining cACGMM and VMFMM models for simultaneous diarization and separation of meeting speech, enabling block-wise processing and improved word error rates.

Contribution

It presents a novel integrated approach for diarization and separation using statistical mixture models, including a new method for counting active speakers per segment.

Findings

Outperforms cascaded methods in WER on LibriCSS corpus

Supports block-online processing with speaker counting

Effectively exploits spatial and spectral information

Abstract

We propose an approach for simultaneous diarization and separation of meeting data. It consists of a complex Angular Central Gaussian Mixture Model (cACGMM) for speech source separation, and a von-Mises-Fisher Mixture Model (VMFMM) for diarization in a joint statistical framework. Through the integration, both spatial and spectral information are exploited for diarization and separation. We also develop a method for counting the number of active speakers in a segment of a meeting to support block-wise processing. While the total number of speakers in a meeting may be known, it is usually not known on a per-segment level. With the proposed speaker counting, joint diarization and source separation can be done segment-by-segment, and the permutation problem across segments is solved, thus allowing for block-online processing in the future. Experimental results on the LibriCSS meeting corpus show that the integrated approach outperforms a cascaded approach of diarization and speech enhancement in terms of WER, both on a per-segment and on a per-meeting level.