Riposte! A Large Corpus of Counter-Arguments

TL;DR

This paper introduces Riposte!, a large-scale corpus of over 18,000 counter-arguments created by crowdworkers, aimed at advancing automatic generation of constructive feedback for fallacious micro-level arguments.

Contribution

The paper presents Riposte!, the first extensive corpus of counter-arguments targeting fallacious micro-arguments, with analysis and baseline models for future research.

Findings

Riposte! contains over 18,000 CAs.

Workers identify fallacy types and generate relevant CAs.

Baseline models are constructed based on corpus analysis.

Abstract

Constructive feedback is an effective method for improving critical thinking skills. Counter-arguments (CAs), one form of constructive feedback, have been proven to be useful for critical thinking skills. However, little work has been done for constructing a large-scale corpus of them which can drive research on automatic generation of CAs for fallacious micro-level arguments (i.e. a single claim and premise pair). In this work, we cast providing constructive feedback as a natural language processing task and create Riposte!, a corpus of CAs, towards this goal. Produced by crowdworkers, Riposte! contains over 18k CAs. We instruct workers to first identify common fallacy types and produce a CA which identifies the fallacy. We analyze how workers create CAs and construct a baseline model based on our analysis.