Enhancing Multi-Domain Automatic Short Answer Grading through an Explainable Neuro-Symbolic Pipeline

TL;DR

This paper introduces a weakly supervised method and a neuro-symbolic model for explainable automatic short answer grading, significantly improving accuracy and providing interpretable reasoning across multiple domains and languages.

Contribution

It proposes a novel weakly supervised annotation process and a neuro-symbolic architecture for explainable grading in ASAG, reducing reliance on annotated justification cues.

Findings

Improved RMSE by 0.24 to 0.3 over state-of-the-art.

Effective in bilingual, multi-domain, multi-question settings.

Provides high-quality grades with explanations.

Abstract

Grading short answer questions automatically with interpretable reasoning behind the grading decision is a challenging goal for current transformer approaches. Justification cue detection, in combination with logical reasoners, has shown a promising direction for neuro-symbolic architectures in ASAG. But, one of the main challenges is the requirement of annotated justification cues in the students' responses, which only exist for a few ASAG datasets. To overcome this challenge, we contribute (1) a weakly supervised annotation procedure for justification cues in ASAG datasets, and (2) a neuro-symbolic model for explainable ASAG based on justification cues. Our approach improves upon the RMSE by 0.24 to 0.3 compared to the state-of-the-art on the Short Answer Feedback dataset in a bilingual, multi-domain, and multi-question training setup. This result shows that our approach provides a promising direction for generating high-quality grades and accompanying explanations for future research in ASAG and educational NLP.