Speech collage: code-switched audio generation by collaging monolingual corpora

TL;DR

This paper presents Speech Collage, a novel method for synthesizing code-switched speech data from monolingual sources to improve automatic speech recognition systems, especially in low-resource scenarios.

Contribution

The paper introduces Speech Collage, a new audio splicing technique for generating code-switched speech data from monolingual corpora, enhancing ASR performance.

Findings

Up to 34.4% reduction in Mixed-Error Rate for in-domain data.

Up to 16.2% reduction in Word-Error Rate for zero-shot scenarios.

CS augmentation increases code-switching tendency and decreases monolingual bias.

Abstract

Designing effective automatic speech recognition (ASR) systems for Code-Switching (CS) often depends on the availability of the transcribed CS resources. To address data scarcity, this paper introduces Speech Collage, a method that synthesizes CS data from monolingual corpora by splicing audio segments. We further improve the smoothness quality of audio generation using an overlap-add approach. We investigate the impact of generated data on speech recognition in two scenarios: using in-domain CS text and a zero-shot approach with synthesized CS text. Empirical results highlight up to 34.4% and 16.2% relative reductions in Mixed-Error Rate and Word-Error Rate for in-domain and zero-shot scenarios, respectively. Lastly, we demonstrate that CS augmentation bolsters the model's code-switching inclination and reduces its monolingual bias.