Autoregressive Score Generation for Multi-trait Essay Scoring

TL;DR

This paper introduces ArTS, a novel autoregressive approach using T5 for multi-trait essay scoring, outperforming previous methods by over 5% in accuracy, and enabling efficient multi-score prediction within a single model.

Contribution

It proposes a new autoregressive score generation model for multi-trait AES that leverages T5, moving beyond traditional regression to improve efficiency and performance.

Findings

Achieved over 5% average improvements in scoring accuracy.

Demonstrated effective multi-trait score prediction with a single model.

Validated the approach's superiority over prior methods.

Abstract

Recently, encoder-only pre-trained models such as BERT have been successfully applied in automated essay scoring (AES) to predict a single overall score. However, studies have yet to explore these models in multi-trait AES, possibly due to the inefficiency of replicating BERT-based models for each trait. Breaking away from the existing sole use of encoder, we propose an autoregressive prediction of multi-trait scores (ArTS), incorporating a decoding process by leveraging the pre-trained T5. Unlike prior regression or classification methods, we redefine AES as a score-generation task, allowing a single model to predict multiple scores. During decoding, the subsequent trait prediction can benefit by conditioning on the preceding trait scores. Experimental results proved the efficacy of ArTS, showing over 5% average improvements in both prompts and traits.