Towards Transparent AI Grading: Semantic Entropy as a Signal for Human-AI Disagreement

TL;DR

This paper proposes semantic entropy, a novel measure based on GPT-4 explanations, to identify uncertainty and disagreement in automated grading, enhancing transparency and trustworthiness across subjects and task types.

Contribution

Introduction of semantic entropy as an interpretable uncertainty signal that correlates with human disagreement and generalizes across subjects and task features.

Findings

Semantic entropy correlates with human grader disagreement.

It varies across academic subjects.

It increases with tasks requiring interpretive reasoning.

Abstract

Automated grading systems can efficiently score short-answer responses, yet they often fail to indicate when a grading decision is uncertain or potentially contentious. We introduce semantic entropy, a measure of variability across multiple GPT-4-generated explanations for the same student response, as a proxy for human grader disagreement. By clustering rationales via entailment-based similarity and computing entropy over these clusters, we quantify the diversity of justifications without relying on final output scores. We address three research questions: (1) Does semantic entropy align with human grader disagreement? (2) Does it generalize across academic subjects? (3) Is it sensitive to structural task features such as source dependency? Experiments on the ASAP-SAS dataset show that semantic entropy correlates with rater disagreement, varies meaningfully across subjects, and increases in tasks requiring interpretive reasoning. Our findings position semantic entropy as an interpretable uncertainty signal that supports more transparent and trustworthy AI-assisted grading workflows.