Experiments of ASR-based mispronunciation detection for children and adult English learners

TL;DR

This paper presents an ASR-based system for detecting mispronunciations in non-native English speakers, focusing on Italian learners, and demonstrates improved accuracy in identifying pronunciation errors using an error model.

Contribution

It introduces a phone-based ASR system with an error model tailored for non-native English pronunciation assessment, validated on Italian adult and child speech corpora.

Findings

Error model improves discrimination of correct and incorrect sounds.

ASR system accuracy increases with the error model.

Effective detection of pronunciation errors in non-native speech.

Abstract

Pronunciation is one of the fundamentals of language learning, and it is considered a primary factor of spoken language when it comes to an understanding and being understood by others. The persistent presence of high error rates in speech recognition domains resulting from mispronunciations motivates us to find alternative techniques for handling mispronunciations. In this study, we develop a mispronunciation assessment system that checks the pronunciation of non-native English speakers, identifies the commonly mispronounced phonemes of Italian learners of English, and presents an evaluation of the non-native pronunciation observed in phonetically annotated speech corpora. In this work, to detect mispronunciations, we used a phone-based ASR implemented using Kaldi. We used two non-native English labeled corpora; (i) a corpus of Italian adults contains 5,867 utterances from 46 speakers, and (ii) a corpus of Italian children consists of 5,268 utterances from 78 children. Our results show that the selected error model can discriminate correct sounds from incorrect sounds in both native and nonnative speech, and therefore can be used to detect pronunciation errors in non-native speech. The phone error rates show improvement in using the error language model. The ASR system shows better accuracy after applying the error model on our selected corpora.