Listen, Correct, and Feed Back: Spoken Pedagogical Feedback Generation

TL;DR

This paper introduces SPFG, a new dataset for spoken pedagogical feedback, and evaluates instruction-tuned LLMs for generating corrective and encouraging feedback in language learning scenarios.

Contribution

It presents a novel dataset with human-verified feedback for language learners and compares fine-tuning and preference alignment methods for feedback generation.

Findings

Supervised fine-tuning yields the most consistent improvements.

Preference-based methods like DPO and KTO show smaller or mixed gains.

Correction quality and feedback quality are weakly correlated.

Abstract

Grammatical error correction (GEC) and explanation (GEE) have made rapid progress, but real teaching scenarios also require \emph{learner-friendly pedagogical feedback} that is actionable, level-appropriate, and encouraging. We introduce \textbf{SPFG} (\textbf{S}poken \textbf{P}edagogical \textbf{F}eedback \textbf{G}eneration), a dataset built based on the Speak \& Improve Challenge 2025 corpus, pairing fluency-oriented transcriptions with GEC targets and \emph{human-verified} teacher-style feedback, including preferred/rejected feedback pairs for preference learning. We study a transcript-based Spoken Grammatical Error Correction (SGEC) setting and evaluate three instruction-tuned LLMs (Qwen2.5, Llama-3.1, and GLM-4), comparing supervised fine-tuning (SFT) with preference-based alignment (using DPO and KTO) for jointly generating corrections and feedback. Results show that SFT provides the most consistent improvements, while DPO/KTO yield smaller or mixed gains, and that correction quality and feedback quality are weakly coupled. Our implementation is available at https://github.com/Skywalker-Harrison/spfg.