Personalised Feedback Framework for Online Education Programmes Using Generative AI

TL;DR

This paper introduces a personalised AI feedback framework for online education that enhances assessment support by integrating embeddings with ChatGPT, achieving high efficacy and rivaling human feedback.

Contribution

It presents a novel feedback system combining embeddings with ChatGPT to provide nuanced, topic-targeted feedback in online education, surpassing previous qualitative benchmarks.

Findings

Achieved 90% efficacy for open-ended questions.

Achieved 100% efficacy for multiple-choice questions.

Framework rivals human feedback quality.

Abstract

AI tools, particularly large language modules, have recently proven their effectiveness within learning management systems and online education programmes. As feedback continues to play a crucial role in learning and assessment in schools, educators must carefully customise the use of AI tools in order to optimally support students in their learning journey. Efforts to improve educational feedback systems have seen numerous attempts reflected in the research studies but mostly have been focusing on qualitatively benchmarking AI feedback against human-generated feedback. This paper presents an exploration of an alternative feedback framework which extends the capabilities of ChatGPT by integrating embeddings, enabling a more nuanced understanding of educational materials and facilitating topic-targeted feedback for quiz-based assessments. As part of the study, we proposed and developed a proof of concept solution, achieving an efficacy rate of 90% and 100% for open-ended and multiple-choice questions, respectively. The results showed that our framework not only surpasses expectations but also rivals human narratives, highlighting the potential of AI in revolutionising educational feedback mechanisms.