Error-preserving Automatic Speech Recognition of Young English Learners' Language

TL;DR

This paper develops an error-preserving automatic speech recognition system tailored for young English learners, enabling accurate transcription of spontaneous speech and retaining learner errors for effective feedback.

Contribution

It introduces a novel ASR model trained on children's speech data that preserves learner errors, addressing limitations of adult-trained models and aiding language learning.

Findings

The model benefits from direct fine-tuning on children's voices.

It achieves a higher error preservation rate than existing models.

The corpus contains 85 hours of learner speech from Switzerland.

Abstract

One of the central skills that language learners need to practice is speaking the language. Currently, students in school do not get enough speaking opportunities and lack conversational practice. Recent advances in speech technology and natural language processing allow for the creation of novel tools to practice their speaking skills. In this work, we tackle the first component of such a pipeline, namely, the automated speech recognition module (ASR), which faces a number of challenges: first, state-of-the-art ASR models are often trained on adult read-aloud data by native speakers and do not transfer well to young language learners' speech. Second, most ASR systems contain a powerful language model, which smooths out errors made by the speakers. To give corrective feedback, which is a crucial part of language learning, the ASR systems in our setting need to preserve the errors made by the language learners. In this work, we build an ASR system that satisfies these requirements: it works on spontaneous speech by young language learners and preserves their errors. For this, we collected a corpus containing around 85 hours of English audio spoken by learners in Switzerland from grades 4 to 6 on different language learning tasks, which we used to train an ASR model. Our experiments show that our model benefits from direct fine-tuning on children's voices and has a much higher error preservation rate than other models.