GradingAttack: Attacking Large Language Models Towards Short Answer Grading Ability

TL;DR

This paper introduces GradingAttack, a framework for adversarially testing large language models used in automatic short answer grading, revealing vulnerabilities that threaten grading fairness and reliability.

Contribution

We propose a novel adversarial attack framework tailored for LLM-based ASAG models, including new token and prompt-level strategies and a camouflage evaluation metric.

Findings

Prompt-level attacks have higher success rates.

Token-level attacks offer better camouflage.

Attacks effectively mislead grading models.

Abstract

Large language models (LLMs) have demonstrated remarkable potential for automatic short answer grading (ASAG), significantly boosting student assessment efficiency and scalability in educational scenarios. However, their vulnerability to adversarial manipulation raises critical concerns about automatic grading fairness and reliability. In this paper, we introduce GradingAttack, a fine-grained adversarial attack framework that systematically evaluates the vulnerability of LLM based ASAG models. Specifically, we align general-purpose attack methods with the specific objectives of ASAG by designing token-level and prompt-level strategies that manipulate grading outcomes while maintaining high camouflage. Furthermore, to quantify attack camouflage, we propose a novel evaluation metric that balances attack success and camouflage. Experiments on multiple datasets demonstrate that both attack strategies effectively mislead grading models, with prompt-level attacks achieving higher success rates and token-level attacks exhibiting superior camouflage capability. Our findings underscore the need for robust defenses to ensure fairness and reliability in ASAG. Our code and datasets are available at https://anonymous.4open.science/r/GradingAttack.