AdaCS: Adaptive Normalization for Enhanced Code-Switching ASR

TL;DR

AdaCS introduces an adaptive normalization model with a bias attention module to improve code-switching speech recognition, especially in low-resource and unseen domain scenarios, achieving significant WER reductions.

Contribution

The paper presents AdaCS, a novel adaptive normalization approach with a bias attention module that enhances code-switching ASR performance across domains.

Findings

Achieves 56.2% WER reduction on Vietnamese CS ASR normalization.

Outperforms previous state-of-the-art methods.

Effectively handles unseen CS phrases across various domains.

Abstract

Intra-sentential code-switching (CS) refers to the alternation between languages that happens within a single utterance and is a significant challenge for Automatic Speech Recognition (ASR) systems. For example, when a Vietnamese speaker uses foreign proper names or specialized terms within their speech. ASR systems often struggle to accurately transcribe intra-sentential CS due to their training on monolingual data and the unpredictable nature of CS. This issue is even more pronounced for low-resource languages, where limited data availability hinders the development of robust models. In this study, we propose AdaCS, a normalization model integrates an adaptive bias attention module (BAM) into encoder-decoder network. This novel approach provides a robust solution to CS ASR in unseen domains, thereby significantly enhancing our contribution to the field. By utilizing BAM to both identify and normalize CS phrases, AdaCS enhances its adaptive capabilities with a biased list of words provided during inference. Our method demonstrates impressive performance and the ability to handle unseen CS phrases across various domains. Experiments show that AdaCS outperforms previous state-of-the-art method on Vietnamese CS ASR normalization by considerable WER reduction of 56.2% and 36.8% on the two proposed test sets.