Prompt Agnostic Essay Scorer: A Domain Generalization Approach to Cross-prompt Automated Essay Scoring
Robert Ridley, Liang He, Xinyu Dai, Shujian Huang, Jiajun Chen

TL;DR
This paper introduces PAES, a prompt-agnostic essay scoring model that performs well across prompts without needing prompt-specific data, advancing the practicality of automated essay scoring systems.
Contribution
The paper presents PAES, a novel single-stage, prompt-agnostic model for cross-prompt AES that does not require labeled or unlabeled target-prompt data during training.
Findings
Achieves state-of-the-art performance on the ASAP dataset.
Does not require prompt-specific data for training.
Simplifies cross-prompt AES application.
Abstract
Cross-prompt automated essay scoring (AES) requires the system to use non target-prompt essays to award scores to a target-prompt essay. Since obtaining a large quantity of pre-graded essays to a particular prompt is often difficult and unrealistic, the task of cross-prompt AES is vital for the development of real-world AES systems, yet it remains an under-explored area of research. Models designed for prompt-specific AES rely heavily on prompt-specific knowledge and perform poorly in the cross-prompt setting, whereas current approaches to cross-prompt AES either require a certain quantity of labelled target-prompt essays or require a large quantity of unlabelled target-prompt essays to perform transfer learning in a multi-step manner. To address these issues, we introduce Prompt Agnostic Essay Scorer (PAES) for cross-prompt AES. Our method requires no access to labelled or unlabelled…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Multimodal Machine Learning Applications · Natural Language Processing Techniques
