Monolingual Data Selection Analysis for English-Mandarin Hybrid Code-switching Speech Recognition

TL;DR

This paper analyzes data selection strategies for building an English-Mandarin code-switching speech recognition system, emphasizing the importance of matching monolingual data and balancing data quantities for optimal performance.

Contribution

It provides insights into effective monolingual data selection and matching for improving code-switching speech recognition accuracy.

Findings

Matching monolingual data to code-switching data improves recognition.

Mandarin data benefits mainly Mandarin-dominated utterances.

Excess monolingual data can degrade system performance.

Abstract

In this paper, we conduct data selection analysis in building an English-Mandarin code-switching (CS) speech recognition (CSSR) system, which is aimed for a real CSSR contest in China. The overall training sets have three subsets, i.e., a code-switching data set, an English (LibriSpeech) and a Mandarin data set respectively. The code-switching data are Mandarin dominated. First of all, it is found using the overall data yields worse results, and hence data selection study is necessary. Then to exploit monolingual data, we find data matching is crucial. Mandarin data is closely matched with the Mandarin part in the code-switching data, while English data is not. However, Mandarin data only helps on those utterances that are significantly Mandarin-dominated. Besides, there is a balance point, over which more monolingual data will divert the CSSR system, degrading results. Finally, we analyze the effectiveness of combining monolingual data to train a CSSR system with the HMM-DNN hybrid framework. The CSSR system can perform within-utterance code-switch recognition, but it still has a margin with the one trained on code-switching data.