Phrase-Level Adversarial Training for Mitigating Bias in Neural Network-based Automatic Essay Scoring

TL;DR

This paper introduces a phrase-level adversarial training method to reduce bias and improve robustness in neural network-based automatic essay scoring systems, demonstrating significant performance gains against adversarial examples.

Contribution

A novel model-agnostic phrase-level adversarial data augmentation technique to mitigate bias and enhance robustness in AES models.

Findings

Improves AES model performance on adversarial samples

Enhances robustness of AES systems against bias

Effective across various neural network architectures

Abstract

Automatic Essay Scoring (AES) is widely used to evaluate candidates for educational purposes. However, due to the lack of representative data, most existing AES systems are not robust, and their scoring predictions are biased towards the most represented data samples. In this study, we propose a model-agnostic phrase-level method to generate an adversarial essay set to address the biases and robustness of AES models. Specifically, we construct an attack test set comprising samples from the original test set and adversarially generated samples using our proposed method. To evaluate the effectiveness of the attack strategy and data augmentation, we conducted a comprehensive analysis utilizing various neural network scoring models. Experimental results show that the proposed approach significantly improves AES model performance in the presence of adversarial examples and scenarios without such attacks.