Large Language Model-Driven Dynamic Assessment of Grammatical Accuracy in English Language Learner Writing

TL;DR

This paper explores using large language models to enhance dynamic assessment of grammatical accuracy in English learners, demonstrating that LLMs like GPT-4o can provide scalable, accurate, and high-quality feedback in language learning.

Contribution

The study develops DynaWrite, a modular system supporting multiple LLMs for dynamic assessment, and identifies GPT-4o as the most effective model for scalable grammatical feedback.

Findings

GPT-4o and neural chat effectively identify grammatical errors.

GPT-4o provides clearer, more explicit hints than neural chat.

GPT-4o demonstrates sufficient speed and stability for real-time assessment.

Abstract

This study investigates the potential for Large Language Models (LLMs) to scale-up Dynamic Assessment (DA). To facilitate such an investigation, we first developed DynaWrite-a modular, microservices-based grammatical tutoring application which supports multiple LLMs to generate dynamic feedback to learners of English. Initial testing of 21 LLMs, revealed GPT-4o and neural chat to have the most potential to scale-up DA in the language learning classroom. Further testing of these two candidates found both models performed similarly in their ability to accurately identify grammatical errors in user sentences. However, GPT-4o consistently outperformed neural chat in the quality of its DA by generating clear, consistent, and progressively explicit hints. Real-time responsiveness and system stability were also confirmed through detailed performance testing, with GPT-4o exhibiting sufficient speed and stability. This study shows that LLMs can be used to scale-up dynamic assessment and thus enable dynamic assessment to be delivered to larger groups than possible in traditional teacher-learner settings.