Intrinsic Quality Assessment of Arguments

TL;DR

This paper investigates the computational assessment of 15 intrinsic quality dimensions of natural language arguments using only textual features, revealing moderate success and highlighting challenges like subjectivity and rhetorical quality.

Contribution

It introduces a systematic study of intrinsic argument quality assessment based solely on text, covering multiple dimensions with experimental validation.

Findings

Moderate but significant learning success for most dimensions

Rhetorical quality is the hardest to assess

Subjectivity features are strongly indicative of quality

Abstract

Several quality dimensions of natural language arguments have been investigated. Some are likely to be reflected in linguistic features (e.g., an argument's arrangement), whereas others depend on context (e.g., relevance) or topic knowledge (e.g., acceptability). In this paper, we study the intrinsic computational assessment of 15 dimensions, i.e., only learning from an argument's text. In systematic experiments with eight feature types on an existing corpus, we observe moderate but significant learning success for most dimensions. Rhetorical quality seems hardest to assess, and subjectivity features turn out strong, although length bias in the corpus impedes full validity. We also find that human assessors differ more clearly to each other than to our approach.