AERA Chat: An Interactive Platform for Automated Explainable Student Answer Assessment

TL;DR

AERA Chat is an interactive platform that uses multiple large language models to score student answers and generate explanations, helping educators evaluate and compare rationale quality effectively.

Contribution

It introduces a novel visualization and evaluation platform for automated explainable student answer assessment leveraging multiple LLMs.

Findings

Effective in evaluating rationale quality across datasets

Facilitates comparison of different rationale-generation methods

Supports educator annotation and assessment tasks

Abstract

Explainability in automated student answer scoring systems is critical for building trust and enhancing usability among educators. Yet, generating high-quality assessment rationales remains challenging due to the scarcity of annotated data and the prohibitive cost of manual verification, prompting heavy reliance on rationales produced by large language models (LLMs), which are often noisy and unreliable. To address these limitations, we present AERA Chat, an interactive visualization platform designed for automated explainable student answer assessment. AERA Chat leverages multiple LLMs to concurrently score student answers and generate explanatory rationales, offering innovative visualization features that highlight critical answer components and rationale justifications. The platform also incorporates intuitive annotation and evaluation tools, supporting educators in marking tasks and researchers in evaluating rationale quality from different models. We demonstrate the effectiveness of our platform through evaluations of multiple rationale-generation methods on several datasets, showcasing its capability for facilitating robust rationale evaluation and comparative analysis.