NTT speaker diarization system for CHiME-7: multi-domain, multi-microphone End-to-end and vector clustering diarization

TL;DR

This paper presents a multi-microphone, multi-domain speaker diarization system combining dereverberation, end-to-end neural diarization, vector clustering, and self-supervised adaptation, achieving significant improvements in CHiME-7 challenge performance.

Contribution

It introduces a novel multi-channel diarization pipeline with self-supervised domain adaptation, enhancing performance over baseline systems in complex conversational environments.

Findings

Achieved 65% and 62% relative improvements over baseline on development and eval sets.

Secured third place in CHiME-7 diarization performance.

Demonstrated effectiveness of self-supervised adaptation in multi-microphone diarization.

Abstract

This paper details our speaker diarization system designed for multi-domain, multi-microphone casual conversations. The proposed diarization pipeline uses weighted prediction error (WPE)-based dereverberation as a front end, then applies end-to-end neural diarization with vector clustering (EEND-VC) to each channel separately. It integrates the diarization result obtained from each channel using diarization output voting error reduction plus overlap (DOVER-LAP). To harness the knowledge from the target domain and results integrated across all channels, we apply self-supervised adaptation for each session by retraining the EEND-VC with pseudo-labels derived from DOVER-LAP. The proposed system was incorporated into NTT's submission for the distant automatic speech recognition task in the CHiME-7 challenge. Our system achieved 65 % and 62 % relative improvements on development and eval sets compared to the organizer-provided VC-based baseline diarization system, securing third place in diarization performance.