Cognitively Aided Zero-Shot Automatic Essay Grading

TL;DR

This paper introduces a zero-shot automatic essay grading method that leverages cognitive gaze data to improve grading accuracy on new prompts, demonstrating significant performance gains.

Contribution

It presents a novel approach using gaze behavior as cognitive information to enhance zero-shot essay grading performance.

Findings

Gaze data improves grading accuracy by nearly 5 percentage points of QWK.

The method is effective for new prompts not seen during training.

Cognitive information enhances zero-shot AEG systems.

Abstract

Automatic essay grading (AEG) is a process in which machines assign a grade to an essay written in response to a topic, called the prompt. Zero-shot AEG is when we train a system to grade essays written to a new prompt which was not present in our training data. In this paper, we describe a solution to the problem of zero-shot automatic essay grading, using cognitive information, in the form of gaze behaviour. Our experiments show that using gaze behaviour helps in improving the performance of AEG systems, especially when we provide a new essay written in response to a new prompt for scoring, by an average of almost 5 percentage points of QWK.