REFINE: Real-world Exploration of Interactive Feedback and Student Behaviour

TL;DR

REFINE is an interactive, multi-agent feedback system using small open-source LLMs that enhances formative feedback by supporting interpretation, clarification, and follow-up in real-world educational settings.

Contribution

This work introduces REFINE, a novel multi-agent system that treats feedback as an interactive process, improving quality and engagement in scalable educational feedback.

Findings

Judge-guided regeneration improves feedback quality.

Interactive agent produces responses comparable to state-of-the-art models.

System feedback influences student inquiry patterns.

Abstract

Formative feedback is central to effective learning, yet providing timely, individualised feedback at scale remains a persistent challenge. While recent work has explored the use of large language models (LLMs) to automate feedback, most existing systems still conceptualise feedback as a static, one-way artifact, offering limited support for interpretation, clarification, or follow-up. In this work, we introduce REFINE, a locally deployable, multi-agent feedback system built on small, open-source LLMs that treats feedback as an interactive process. REFINE combines a pedagogically-grounded feedback generation agent with an LLM-as-a-judge-guided regeneration loop using a human-aligned judge, and a self-reflective tool-calling interactive agent that supports student follow-up questions with context-aware, actionable responses. We evaluate REFINE through controlled experiments and an authentic classroom deployment in an undergraduate computer science course. Automatic evaluations show that judge-guided regeneration significantly improves feedback quality, and that the interactive agent produces efficient, high-quality responses comparable to a state-of-the-art closed-source model. Analysis of real student interactions further reveals distinct engagement patterns and indicates that system-generated feedback systematically steers subsequent student inquiry. Our findings demonstrate the feasibility and effectiveness of multi-agent, tool-augmented feedback systems for scalable, interactive feedback.