SEFL: A Framework for Generating Synthetic Educational Assignment Feedback with LLM Agents

TL;DR

SEFL introduces a synthetic data framework using LLMs to generate high-quality educational feedback at scale, reducing reliance on real student data and enabling efficient model fine-tuning for improved feedback quality.

Contribution

The paper presents a novel synthetic data generation framework, SEFL, that leverages LLMs to simulate educational feedback, facilitating scalable and effective training of feedback models.

Findings

SEFL-generated feedback outperforms baseline models in quality.

Synthetic data enables efficient fine-tuning of smaller LLMs.

Human evaluations confirm improved feedback relevance and usefulness.

Abstract

Providing high-quality feedback on student assignments is crucial for student success, but it is heavily limited by time and budgetary constraints. In this work, we introduce Synthetic Educational Feedback Loops (SEFL), a synthetic data framework designed to generate data that resembles immediate, on-demand feedback at scale without relying on extensive, real-world student assignments and teacher feedback. To obtain this type of data, two large language models (LLMs) operate in a teacher-student role to simulate assignment completion and formative feedback, generating 19.8K synthetic pairs of student work and corresponding critiques and actionable improvements from a teacher. With this data, we fine-tune smaller, more computationally efficient LLMs on these synthetic pairs, enabling them to replicate key features of high-quality, goal-oriented feedback. Through comprehensive evaluations with three LLM judges and three human experts, across a subset of 900 outputs, we demonstrate that SEFL-tuned models outperform both their untuned counterparts and an existing baseline in terms of feedback quality. The potential for societal impact is reinforced by extensive qualitative comments and ratings from human stakeholders -- both students and higher education instructors. SEFL has the potential to transform feedback processes for higher education and beyond.