Composable Cross-prompt Essay Scoring by Merging Models

TL;DR

This paper introduces a source-free, model-merging approach for cross-prompt essay scoring that outperforms joint training methods, enhances robustness under distribution shifts, and preserves computational efficiency.

Contribution

It proposes a novel source-free adaptation method using linear combinations of trained models and a new optimization objective, PIM, for improved cross-prompt essay scoring.

Findings

Outperforms joint training on all sources in accuracy

Maintains robustness under distribution shifts

Effective in severe domain adaptation scenarios

Abstract

Recent advances in cross-prompt automated essay scoring (AES) typically train models jointly on all source prompts, often requiring additional access to unlabeled target prompt essays simultaneously. However, using all sources is suboptimal in our pilot study, and re-accessing source datasets during adaptation raises privacy concerns. We propose a source-free adaptation approach that selectively merges individually trained source models' parameters instead of datasets. In particular, we simulate joint training through linear combinations of task vectors -- the parameter updates from fine-tuning. To optimize the combination's coefficients, we propose Prior-encoded Information Maximization (PIM), an unsupervised objective which promotes the model's score discriminability regularized by priors pre-computed from the sources. We employ Bayesian optimization as an efficient optimizer of PIM. Experimental results with LLMs on in-dataset and cross-dataset adaptation show that our method (1) consistently outperforms training jointly on all sources, (2) maintains superior robustness compared to other merging methods, (3) excels under severe distribution shifts where recent leading cross-prompt methods struggle, all while retaining computational efficiency.