Arabic Code-Switching Speech Recognition using Monolingual Data

TL;DR

This paper investigates multilingual ASR for Arabic-English code-switching, comparing WFST and transformer approaches, and introduces new datasets for benchmarking performance in different code-switching scenarios.

Contribution

It presents a novel multi-graph WFST decoding framework and provides comprehensive experiments comparing it with transformer models for code-switching speech recognition.

Findings

WFST decoding outperforms transformers in intersentential code-switching.

Transformers excel in intrasentential code-switching.

New datasets enable benchmarking of code-switching ASR systems.

Abstract

Code-switching in automatic speech recognition (ASR) is an important challenge due to globalization. Recent research in multilingual ASR shows potential improvement over monolingual systems. We study key issues related to multilingual modeling for ASR through a series of large-scale ASR experiments. Our innovative framework deploys a multi-graph approach in the weighted finite state transducers (WFST) framework. We compare our WFST decoding strategies with a transformer sequence to sequence system trained on the same data. Given a code-switching scenario between Arabic and English languages, our results show that the WFST decoding approaches were more suitable for the intersentential code-switching datasets. In addition, the transformer system performed better for intrasentential code-switching task. With this study, we release an artificially generated development and test sets, along with ecological code-switching test set, to benchmark the ASR performance.