Automated Genre-Aware Article Scoring and Feedback Using Large Language Models

TL;DR

This paper presents an advanced automated article scoring system that leverages BERT and Chat-GPT to evaluate and provide detailed, genre-specific feedback, outperforming traditional methods and aiding educational writing improvement.

Contribution

It introduces a novel integrated system combining BERT and Chat-GPT for genre-aware scoring and personalized feedback, enhancing accuracy and usefulness over existing approaches.

Findings

Outperforms traditional scoring methods on multiple datasets

Provides detailed feature-based assessments for various genres

Generates personalized feedback to improve writing skills

Abstract

This paper focuses on the development of an advanced intelligent article scoring system that not only assesses the overall quality of written work but also offers detailed feature-based scoring tailored to various article genres. By integrating the pre-trained BERT model with the large language model Chat-GPT, the system gains a deep understanding of both the content and structure of the text, enabling it to provide a thorough evaluation along with targeted suggestions for improvement. Experimental results demonstrate that this system outperforms traditional scoring methods across multiple public datasets, particularly in feature-based assessments, offering a more accurate reflection of the quality of different article types. Moreover, the system generates personalized feedback to assist users in enhancing their writing skills, underscoring the potential and practical value of automated scoring technologies in educational contexts.