Happy Are Those Who Grade without Seeing: A Multi-Task Learning Approach to Grade Essays Using Gaze Behaviour

TL;DR

This paper introduces a multi-task learning approach that leverages gaze behaviour to enhance automatic essay grading, reducing the need for costly gaze data collection while achieving significant performance improvements.

Contribution

It presents a novel multi-task learning framework that learns gaze behaviour at runtime to improve essay grading accuracy without extensive gaze data collection.

Findings

Significant improvement over state-of-the-art systems with gaze data.

Effective performance gains even without available gaze data.

Demonstrated scalability across multiple essay sets.

Abstract

The gaze behaviour of a reader is helpful in solving several NLP tasks such as automatic essay grading. However, collecting gaze behaviour from readers is costly in terms of time and money. In this paper, we propose a way to improve automatic essay grading using gaze behaviour, which is learnt at run time using a multi-task learning framework. To demonstrate the efficacy of this multi-task learning based approach to automatic essay grading, we collect gaze behaviour for 48 essays across 4 essay sets, and learn gaze behaviour for the rest of the essays, numbering over 7000 essays. Using the learnt gaze behaviour, we can achieve a statistically significant improvement in performance over the state-of-the-art system for the essay sets where we have gaze data. We also achieve a statistically significant improvement for 4 other essay sets, numbering about 6000 essays, where we have no gaze behaviour data available. Our approach establishes that learning gaze behaviour improves automatic essay grading.