Do We Still Need Audio? Rethinking Speaker Diarization with a Text-Based Approach Using Multiple Prediction Models

TL;DR

This paper introduces a text-based speaker diarization method using sentence-level change detection, which outperforms traditional audio-based systems in short conversations and emphasizes semantic features.

Contribution

The paper proposes two novel text-based models for speaker diarization, demonstrating competitive performance and highlighting the potential of linguistic features over audio cues.

Findings

MPM outperforms audio-based SD in short conversations

Text-based SD achieves competitive accuracy with state-of-the-art systems

Semantic features enhance speaker change detection

Abstract

We present a novel approach to Speaker Diarization (SD) by leveraging text-based methods focused on Sentence-level Speaker Change Detection within dialogues. Unlike audio-based SD systems, which are often challenged by audio quality and speaker similarity, our approach utilizes the dialogue transcript alone. Two models are developed: the Single Prediction Model (SPM) and the Multiple Prediction Model (MPM), both of which demonstrate significant improvements in identifying speaker changes, particularly in short conversations. Our findings, based on a curated dataset encompassing diverse conversational scenarios, reveal that the text-based SD approach, especially the MPM, performs competitively against state-of-the-art audio-based SD systems, with superior performance in short conversational contexts. This paper not only showcases the potential of leveraging linguistic features for SD but also highlights the importance of integrating semantic understanding into SD systems, opening avenues for future research in multimodal and semantic feature-based diarization.