Mitigating Bias in Automated Grading Systems for ESL Learners: A Contrastive Learning Approach

TL;DR

This paper addresses bias in automated ESL essay scoring by introducing a contrastive learning method that reduces scoring disparities between native and ESL writers, improving fairness without sacrificing accuracy.

Contribution

It proposes a novel contrastive learning approach with matched essay pairs to mitigate bias in transformer-based AES models for ESL learners.

Findings

Reduced ESL-native scoring gap by 39.9%

Maintained high model agreement with QWK of 0.76

Disentangled sentence complexity from grammatical errors

Abstract

As Automated Essay Scoring (AES) systems are increasingly used in high-stakes educational settings, concerns regarding algorithmic bias against English as a Second Language (ESL) learners have increased. Current Transformer-based regression models trained primarily on native-speaker corpora often learn spurious correlations between surface-level L2 linguistic features and essay quality. In this study, we conduct a bias study of a fine-tuned DeBERTa-v3 model using the ASAP 2.0 and ELLIPSE datasets, revealing a constrained score scaling for high-proficiency ESL writing where high-proficiency ESL essays receive scores 10.3% lower than Native speaker essays of identical human-rated quality. To mitigate this, we propose applying contrastive learning with a triplet construction strategy: Contrastive Learning with Matched Essay Pairs. We constructed a dataset of 17,161 matched essay pairs and fine-tuned the model using Triplet Margin Loss to align the latent representations of ESL and Native writing. Our approach reduced the high-proficiency scoring disparity by 39.9% (to a 6.2% gap) while maintaining a Quadratic Weighted Kappa (QWK) of 0.76. Post-hoc linguistic analysis suggests the model successfully disentangled sentence complexity from grammatical error, preventing the penalization of valid L2 syntactic structures.