Token-level Speaker Change Detection Using Speaker Difference and Speech Content via Continuous Integrate-and-fire

TL;DR

This paper introduces a novel token-level speaker change detection method that combines speaker difference and speech content cues using the continuous integrate-and-fire mechanism, improving accuracy over frame-level systems.

Contribution

It proposes a new SCD approach that integrates speech content and speaker difference cues at token-level, utilizing CIF for better boundary detection.

Findings

Outperforms frame-level baseline by 2.45% ECP.

Highlights importance of speech content in SCD.

Demonstrates advantages of token-level detection over frame-level.

Abstract

In multi-talker scenarios such as meetings and conversations, speech processing systems are usually required to segment the audio and then transcribe each segmentation. These two stages are addressed separately by speaker change detection (SCD) and automatic speech recognition (ASR). Most previous SCD systems rely solely on speaker information and ignore the importance of speech content. In this paper, we propose a novel SCD system that considers both cues of speaker difference and speech content. These two cues are converted into token-level representations by the continuous integrate-and-fire (CIF) mechanism and then combined for detecting speaker changes on the token acoustic boundaries. We evaluate the performance of our approach on a public real-recorded meeting dataset, AISHELL-4. The experiment results show that our method outperforms a competitive frame-level baseline system by 2.45% equal coverage-purity (ECP). In addition, we demonstrate the importance of speech content and speaker difference to the SCD task, and the advantages of conducting SCD on the token acoustic boundaries compared with conducting SCD frame by frame.