Improve LLM-based Automatic Essay Scoring with Linguistic Features

TL;DR

This paper enhances LLM-based automatic essay scoring by integrating linguistic features, resulting in improved accuracy across diverse prompts while maintaining computational efficiency.

Contribution

It introduces a hybrid approach that combines linguistic features with LLMs, outperforming baseline models in both in-domain and out-of-domain scenarios.

Findings

Hybrid model outperforms baseline in accuracy

Improved performance on diverse prompts

Maintains computational efficiency during inference

Abstract

Automatic Essay Scoring (AES) assigns scores to student essays, reducing the grading workload for instructors. Developing a scoring system capable of handling essays across diverse prompts is challenging due to the flexibility and diverse nature of the writing task. Existing methods typically fall into two categories: supervised feature-based approaches and large language model (LLM)-based methods. Supervised feature-based approaches often achieve higher performance but require resource-intensive training. In contrast, LLM-based methods are computationally efficient during inference but tend to suffer from lower performance. This paper combines these approaches by incorporating linguistic features into LLM-based scoring. Experimental results show that this hybrid method outperforms baseline models for both in-domain and out-of-domain writing prompts.