CS-Dialogue: A 104-Hour Dataset of Spontaneous Mandarin-English Code-Switching Dialogues for Speech Recognition

TL;DR

This paper introduces CS-Dialogue, a large-scale, 104-hour Mandarin-English code-switching speech dataset with full-length dialogues and transcriptions, aiming to advance ASR systems for naturalistic conversational code-switching scenarios.

Contribution

The paper presents a novel, extensive dataset of spontaneous Mandarin-English code-switching dialogues with complete transcriptions, filling a gap in resources for robust ASR development.

Findings

State-of-the-art models struggle with code-switching recognition.

Pre-trained models like Whisper have room for improvement.

Benchmark results highlight the dataset's complexity for ASR.

Abstract

Code-switching (CS), the alternation between two or more languages within a single conversation, presents significant challenges for automatic speech recognition (ASR) systems. Existing Mandarin-English code-switching datasets often suffer from limitations in size, spontaneity, and the lack of full-length dialogue recordings with transcriptions, hindering the development of robust ASR models for real-world conversational scenarios. This paper introduces CS-Dialogue, a novel large-scale Mandarin-English code-switching speech dataset comprising 104 hours of spontaneous conversations from 200 speakers. Unlike previous datasets, CS-Dialogue provides full-length dialogue recordings with complete transcriptions, capturing naturalistic code-switching patterns in continuous speech. We describe the data collection and annotation processes, present detailed statistics of the dataset, and establish benchmark ASR performance using state-of-the-art models. Our experiments, using Transformer, Conformer, and Branchformer, demonstrate the challenges of code-switching ASR, and show that existing pre-trained models such as Whisper still have the space to improve. The CS-Dialogue dataset will be made freely available for all academic purposes.