Reducing Spelling Inconsistencies in Code-Switching ASR using Contextualized CTC Loss

TL;DR

This paper introduces a novel Contextualized CTC loss for character-based code-switching ASR, improving spelling consistency and performance without needing frame-level alignments, thus enhancing multilingual speech recognition accuracy.

Contribution

The paper proposes the CCTC loss that enforces spelling consistency in character-based ASR models for code-switching, without requiring frame-level alignments, and demonstrates improved performance.

Findings

Improved ASR accuracy on code-switching and monolingual data.

Enhanced spelling consistency in character-based models.

No need for frame-level alignments with CCTC loss.

Abstract

Code-Switching (CS) remains a challenge for Automatic Speech Recognition (ASR), especially character-based models. With the combined choice of characters from multiple languages, the outcome from character-based models suffers from phoneme duplication, resulting in language-inconsistent spellings. We propose Contextualized Connectionist Temporal Classification (CCTC) loss to encourage spelling consistencies of a character-based non-autoregressive ASR which allows for faster inference. The CCTC loss conditions the main prediction on the predicted contexts to ensure language consistency in the spellings. In contrast to existing CTC-based approaches, CCTC loss does not require frame-level alignments, since the context ground truth is obtained from the model's estimated path. Compared to the same model trained with regular CTC loss, our method consistently improved the ASR performance on both CS and monolingual corpora.