An Automated Explainable Educational Assessment System Built on LLMs

TL;DR

This paper introduces AERA Chat, an automated, explainable educational assessment system utilizing large language models to provide interactive, visual evaluations, rationales, and insights into student responses, improving transparency and efficiency.

Contribution

The paper presents a novel system that combines LLMs with explainability and visualization tools for automated educational assessment, addressing explainability and annotation cost challenges.

Findings

Provides automated marking with rationale explanations

Enhances assessment transparency with visualization tools

Facilitates efficient rationale verification

Abstract

In this demo, we present AERA Chat, an automated and explainable educational assessment system designed for interactive and visual evaluations of student responses. This system leverages large language models (LLMs) to generate automated marking and rationale explanations, addressing the challenge of limited explainability in automated educational assessment and the high costs associated with annotation. Our system allows users to input questions and student answers, providing educators and researchers with insights into assessment accuracy and the quality of LLM-assessed rationales. Additionally, it offers advanced visualization and robust evaluation tools, enhancing the usability for educational assessment and facilitating efficient rationale verification. Our demo video can be found at https://youtu.be/qUSjz-sxlBc.