Evaluation of Large Language Models' educational feedback in Higher Education: potential, limitations and implications for educational practice

TL;DR

This study evaluates how large language models can generate structured educational feedback in higher education, highlighting their potential and limitations for supporting formative learning when guided by clear instructions.

Contribution

It provides an empirical assessment of LLMs' ability to produce effective feedback in educational settings, using a structured analytical framework.

Findings

LLMs can generate well-structured feedback

Effective feedback depends on clear contextual guidance

Potential for sustainable feedback tools in education

Abstract

The importance of managing feedback practices in higher education has been widely recognised, as they play a crucial role in enhancing teaching, learning, and assessment processes. In today's educational landscape, feedback practices are increasingly influenced by technological advancements, particularly artificial intelligence (AI). Understanding the impact of AI on feedback generation is essential for identifying its potential benefits and establishing effective implementation strategies. This study examines how AI-generated feedback supports student learning using a well-established analytical framework. Specifically, feedback produced by different Large Language Models (LLMs) was assessed in relation to student-designed projects within a training course on inclusive teaching and learning. The evaluation process involved providing seven LLMs with a structured rubric, developed by the university instructor, which defined specific criteria and performance levels. The LLMs were tasked with generating both quantitative assessments and qualitative feedback based on this rubric. The AI-generated feedback was then analysed using Hughes, Smith, and Creese's framework to evaluate its structure and effectiveness in fostering formative learning experiences. Overall, these findings indicate that LLMs can generate well-structured feedback and hold great potential as a sustainable and meaningful feedback tool, provided they are guided by clear contextual information and a well-defined instructions that will be explored further in the conclusions.