Building Robust and Scalable Multilingual ASR for Indian Languages

TL;DR

This paper presents a novel multilingual ASR system for Indian languages that improves language and dialect prediction accuracy using a multi-decoder architecture and phonemic intermediate representations, outperforming baselines in several languages.

Contribution

Introduces a Multi-Decoder architecture with phonemic Common Label Set for multilingual ASR, enhancing performance without additional data, and achieves state-of-the-art results in language and dialect identification.

Findings

Outperforms baseline in 3 languages in WER/CER

Achieves highest language and dialect ID accuracy among participants

Effective phonemic-to-grapheme conversion methods developed

Abstract

This paper describes the systems developed by SPRING Lab, Indian Institute of Technology Madras, for the ASRU MADASR 2.0 challenge. The systems developed focuses on adapting ASR systems to improve in predicting the language and dialect of the utterance among 8 languages across 33 dialects. We participated in Track 1 and Track 2, which restricts the use of additional data and develop from-the-scratch multilingual systems. We presented a novel training approach using Multi-Decoder architecture with phonemic Common Label Set (CLS) as intermediate representation. It improved the performance over the baseline (in the CLS space). We also discuss various methods used to retain the gain obtained in the phonemic space while converting them back to the corresponding grapheme representations. Our systems beat the baseline in 3 languages (Track 2) in terms of WER/CER and achieved the highest language ID and dialect ID accuracy among all participating teams (Track 2).