Scaling and Prompting for Improved End-to-End Spoken Grammatical Error Correction

TL;DR

This paper explores enhancing end-to-end spoken grammatical error correction by expanding training data with pseudo-labelling, using prompting techniques, and analyzing the effects of model size on performance.

Contribution

It introduces a pseudo-labelling process to significantly increase training data and evaluates prompting methods for improved SGEC and feedback generation.

Findings

Pseudo-labelling expands training data from 77 to 2500 hours.

Prompting improves feedback generation more than SGEC performance.

Larger models benefit less from pseudo-labelled data but gain from prompting.

Abstract

Spoken Grammatical Error Correction (SGEC) and Feedback (SGECF) are crucial for second language learners, teachers and test takers. Traditional SGEC systems rely on a cascaded pipeline consisting of an ASR, a module for disfluency detection (DD) and removal and one for GEC. With the rise of end-to-end (E2E) speech foundation models, we investigate their effectiveness in SGEC and feedback generation. This work introduces a pseudo-labelling process to address the challenge of limited labelled data, expanding the training data size from 77 hours to approximately 2500 hours, leading to improved performance. Additionally, we prompt an E2E Whisper-based SGEC model with fluent transcriptions, showing a slight improvement in SGEC performance, with more significant gains in feedback generation. Finally, we assess the impact of increasing model size, revealing that while pseudo-labelled data does not yield performance gain for a larger Whisper model, training with prompts proves beneficial.