Automated Essay Scoring in Argumentative Writing: DeBERTeachingAssistant

TL;DR

This paper introduces a transformer-based model that assesses the persuasiveness of argumentative essays with above-human accuracy, aiming to provide actionable feedback and enhance educational tools.

Contribution

It presents a novel transformer architecture capable of evaluating persuasiveness in argumentative essays, addressing a gap in existing automated scoring systems.

Findings

Achieves above-human accuracy in annotating argumentative discourse elements.

Focuses on assessing persuasiveness, a less explored aspect in automated scoring.

Lays groundwork for explainability to improve feedback to students.

Abstract

Automated Essay scoring has been explored as a research and industry problem for over 50 years. It has drawn a lot of attention from the NLP community because of its clear educational value as a research area that can engender the creation of valuable time-saving tools for educators around the world. Yet, these tools are generally focused on detecting good grammar, spelling mistakes, and organization quality but tend to fail at incorporating persuasiveness features in their final assessment. The responsibility to give actionable feedback to the student to improve the strength of their arguments is left solely on the teacher's shoulders. In this work, we present a transformer-based architecture capable of achieving above-human accuracy in annotating argumentative writing discourse elements for their persuasiveness quality and we expand on planned future work investigating the explainability of our model so that actionable feedback can be offered to the student and thus potentially enable a partnership between the teacher's advice and the machine's advice.