Corpus Wide Argument Mining -- a Working Solution

TL;DR

This paper introduces a comprehensive argument mining system that retrieves relevant argumentative content across a large corpus of newspaper articles, using sentence-level queries and an iterative annotation scheme to improve precision.

Contribution

It presents the first end-to-end high-precision argument mining system capable of handling a wide range of topics in a large corpus, combining sentence queries with iterative annotation.

Findings

Achieved high-precision retrieval of arguments across a large newspaper corpus

Developed an iterative annotation scheme to address label bias

Enabled comprehensive argument mining over diverse topics

Abstract

One of the main tasks in argument mining is the retrieval of argumentative content pertaining to a given topic. Most previous work addressed this task by retrieving a relatively small number of relevant documents as the initial source for such content. This line of research yielded moderate success, which is of limited use in a real-world system. Furthermore, for such a system to yield a comprehensive set of relevant arguments, over a wide range of topics, it requires leveraging a large and diverse corpus in an appropriate manner. Here we present a first end-to-end high-precision, corpus-wide argument mining system. This is made possible by combining sentence-level queries over an appropriate indexing of a very large corpus of newspaper articles, with an iterative annotation scheme. This scheme addresses the inherent label bias in the data and pinpoints the regions of the sample space whose manual labeling is required to obtain high-precision among top-ranked candidates.