An Exploratory Study of Argumentative Writing by Young Students: A Transformer-based Approach

TL;DR

This study explores using transformer models to analyze argumentative writing by middle school students, showing significant improvements over traditional lexical features and revealing structural similarities with adult writing.

Contribution

It demonstrates that transformer-based models significantly outperform lexical features in analyzing young students' argumentative critiques and uncovers structural similarities with adult writing.

Findings

Transformer models improve critique detection by over 20% F1 score.

Lexical and discourse features underperform on young students' data.

Children's writing shares basic local structures with mature writers.

Abstract

We present a computational exploration of argument critique writing by young students. Middle school students were asked to criticize an argument presented in the prompt, focusing on identifying and explaining the reasoning flaws. This task resembles an established college-level argument critique task. Lexical and discourse features that utilize detailed domain knowledge to identify critiques exist for the college task but do not perform well on the young students data. Instead, transformer-based architecture (e.g., BERT) fine-tuned on a large corpus of critique essays from the college task performs much better (over 20% improvement in F1 score). Analysis of the performance of various configurations of the system suggests that while children's writing does not exhibit the standard discourse structure of an argumentative essay, it does share basic local sequential structures with the more mature writers.