Enhancing Automated Essay Scoring with Three Techniques: Two-Stage Fine-Tuning, Score Alignment, and Self-Training

TL;DR

This paper introduces three techniques—Two-Stage fine-tuning, Score Alignment, and Self-Training—to significantly improve automated essay scoring, especially in scenarios with limited labeled data, achieving near full-data performance with fewer samples.

Contribution

The paper presents a novel combination of three techniques to enhance AES performance, particularly under data scarcity, and demonstrates their effectiveness on the ASAP++ dataset.

Findings

All three techniques improve performance in limited-data settings.

Score Alignment enhances consistency and achieves state-of-the-art results.

Integration of techniques reaches 91.2% of full-data performance with 32 samples.

Abstract

Automated Essay Scoring (AES) plays a crucial role in education by providing scalable and efficient assessment tools. However, in real-world settings, the extreme scarcity of labeled data severely limits the development and practical adoption of robust AES systems. This study proposes a novel approach to enhance AES performance in both limited-data and full-data settings by introducing three key techniques. First, we introduce a Two-Stage fine-tuning strategy that leverages low-rank adaptations to better adapt an AES model to target prompt essays. Second, we introduce a Score Alignment technique to improve consistency between predicted and true score distributions. Third, we employ uncertainty-aware self-training using unlabeled data, effectively expanding the training set with pseudo-labeled samples while mitigating label noise propagation. We implement above three key techniques on DualBERT. We conduct extensive experiments on the ASAP++ dataset. As a result, in the 32-data setting, all three key techniques improve performance, and their integration achieves 91.2% of the full-data performance trained on approximately 1,000 labeled samples. In addition, the proposed Score Alignment technique consistently improves performance in both limited-data and full-data settings: e.g., it achieves state-of-the-art results in the full-data setting when integrated into DualBERT.