Pronunciation Modeling of Foreign Words for Mandarin ASR by Considering the Effect of Language Transfer

TL;DR

This paper introduces lexical rules to model language transfer effects, enabling Mandarin ASR systems to recognize English words more accurately without retraining, by augmenting the lexicon with phonetic representations of foreign words.

Contribution

It proposes a set of lexical rules to convert English words into Mandarin phonetic forms, enhancing ASR performance on mixed speech without retraining.

Findings

Improved recognition accuracy for English words in Mandarin ASR

No degradation in Mandarin-only speech recognition

Rules generalize to unseen English words

Abstract

One of the challenges in automatic speech recognition is foreign words recognition. It is observed that a speaker's pronunciation of a foreign word is influenced by his native language knowledge, and such phenomenon is known as the effect of language transfer. This paper focuses on examining the phonetic effect of language transfer in automatic speech recognition. A set of lexical rules is proposed to convert an English word into Mandarin phonetic representation. In this way, a Mandarin lexicon can be augmented by including English words. Hence, the Mandarin ASR system becomes capable to recognize English words without retraining or re-estimation of the acoustic model parameters. Using the lexicon that derived from the proposed rules, the ASR performance of Mandarin English mixed speech is improved without harming the accuracy of Mandarin only speech. The proposed lexical rules are generalized and they can be directly applied to unseen English words.