An LLM-Based System for Argument Mining

TL;DR

This paper introduces an LLM-based system that reconstructs argument structures from text into directed graphs, demonstrating effective recovery of argumentative components and relations through manual and benchmark evaluations.

Contribution

The work presents a novel end-to-end LLM pipeline for argument mining that identifies components and relations, and evaluates its performance on multiple datasets.

Findings

System effectively recovers argumentative structures from text.

Achieves reasonable performance across benchmark datasets.

Demonstrates potential for scalable argument mining using LLMs.

Abstract

Arguments are a fundamental aspect of human reasoning, in which claims are supported, challenged, and weighed against one another. We present an end-to-end large language model (LLM)-based system for reconstructing arguments from natural language text into abstract argument graphs. The system follows a multi-stage pipeline that progressively identifies argumentative components, selects relevant elements, and uncovers their logical relations. These elements are represented as directed acyclic graphs consisting of two component types (premises and conclusions) and three relation types (support, attack, and undercut). We conduct two complementary experiments to evaluate the system. First, we perform a manual evaluation on arguments drawn from an argumentation theory textbook to assess the system's ability to recover argumentative structure. Second, we conduct a quantitative evaluation on benchmark datasets, allowing comparison with prior work by mapping our outputs to established annotation schemes. Results show that the system can adequately recover argumentative structures and, when adapted to different annotation schemes, achieve reasonable performance across benchmark datasets. These findings highlight the potential of LLM-based pipelines for scalable argument mining.