End-to-End ASR for Code-switched Hindi-English Speech

TL;DR

This paper investigates end-to-end speech recognition for code-switched Hindi-English with limited data, employing multi-task learning and corpus balancing to enhance performance over traditional systems.

Contribution

It introduces methods like multi-task learning and corpus balancing to improve low-resource end-to-end ASR for code-switched speech.

Findings

Multi-task learning improves recognition accuracy.

Corpus balancing mitigates class imbalance issues.

Promising results compared to traditional systems.

Abstract

End-to-end (E2E) models have been explored for large speech corpora and have been found to match or outperform traditional pipeline-based systems in some languages. However, most prior work on end-to-end models use speech corpora exceeding hundreds or thousands of hours. In this study, we explore end-to-end models for code-switched Hindi-English language with less than 50 hours of data. We utilize two specific measures to improve network performance in the low-resource setting, namely multi-task learning (MTL) and balancing the corpus to deal with the inherent class imbalance problem i.e. the skewed frequency distribution over graphemes. We compare the results of the proposed approaches with traditional, cascaded ASR systems. While the lack of data adversely affects the performance of end-to-end models, we see promising improvements with MTL and balancing the corpus.