End-to-End Code Switching Language Models for Automatic Speech Recognition

TL;DR

This paper develops end-to-end language models for automatic speech recognition that effectively handle code-switched speech, improving monolingual text extraction and robustness in bilingual communities.

Contribution

It introduces a novel approach combining deep bi-directional language models and machine translation for better code-switched speech recognition.

Findings

Improved WER and perplexity metrics for code-switched speech

Effective extraction of monolingual text from code-switched speech

Enhanced robustness of ASR models in bilingual contexts

Abstract

In this paper, we particularly work on the code-switched text, one of the most common occurrences in the bilingual communities across the world. Due to the discrepancies in the extraction of code-switched text from an Automated Speech Recognition(ASR) module, and thereby extracting the monolingual text from the code-switched text, we propose an approach for extracting monolingual text using Deep Bi-directional Language Models(LM) such as BERT and other Machine Translation models, and also explore different ways of extracting code-switched text from the ASR model. We also explain the robustness of the model by comparing the results of Perplexity and other different metrics like WER, to the standard bi-lingual text output without any external information.