Reading Miscue Detection in Primary School through Automatic Speech Recognition

TL;DR

This paper evaluates the effectiveness of state-of-the-art pretrained ASR models in recognizing Dutch children's speech and detecting reading miscues, highlighting the potential for improved reading diagnosis tools.

Contribution

It compares multiple advanced ASR models for child speech recognition and miscue detection in Dutch, identifying the most effective models and their performance metrics.

Findings

Hubert Large achieves state-of-the-art phoneme-level recognition with PER 23.1%.

Whisper achieves state-of-the-art word-level recognition with WER 9.8%.

Wav2Vec2 Large has the highest recall (0.83), Whisper has the highest precision (0.52).

Abstract

Automatic reading diagnosis systems can benefit both teachers for more efficient scoring of reading exercises and students for accessing reading exercises with feedback more easily. However, there are limited studies on Automatic Speech Recognition (ASR) for child speech in languages other than English, and limited research on ASR-based reading diagnosis systems. This study investigates how efficiently state-of-the-art (SOTA) pretrained ASR models recognize Dutch native children speech and manage to detect reading miscues. We found that Hubert Large finetuned on Dutch speech achieves SOTA phoneme-level child speech recognition (PER at 23.1\%), while Whisper (Faster Whisper Large-v2) achieves SOTA word-level performance (WER at 9.8\%). Our findings suggest that Wav2Vec2 Large and Whisper are the two best ASR models for reading miscue detection. Specifically, Wav2Vec2 Large shows the highest recall at 0.83, whereas Whisper exhibits the highest precision at 0.52 and an F1 score of 0.52.