WIBA: What Is Being Argued? A Comprehensive Approach to Argument Mining

TL;DR

WIBA introduces a comprehensive framework utilizing fine-tuned large language models to detect arguments, identify their topics, and classify their stances across diverse texts, advancing argument mining capabilities.

Contribution

The paper presents a novel, integrated approach for argument detection, topic identification, and stance classification using LLMs, with new models and benchmarks for each task.

Findings

Argument detection F1 score between 79% and 86%.

Topic identification similarity score of 71%, outperforming naive methods.

Stance classification F1 score between 71% and 78%.

Abstract

We propose WIBA, a novel framework and suite of methods that enable the comprehensive understanding of "What Is Being Argued" across contexts. Our approach develops a comprehensive framework that detects: (a) the existence, (b) the topic, and (c) the stance of an argument, correctly accounting for the logical dependence among the three tasks. Our algorithm leverages the fine-tuning and prompt-engineering of Large Language Models. We evaluate our approach and show that it performs well in all the three capabilities. First, we develop and release an Argument Detection model that can classify a piece of text as an argument with an F1 score between 79% and 86% on three different benchmark datasets. Second, we release a language model that can identify the topic being argued in a sentence, be it implicit or explicit, with an average similarity score of 71%, outperforming current naive methods by nearly 40%. Finally, we develop a method for Argument Stance Classification, and evaluate the capability of our approach, showing it achieves a classification F1 score between 71% and 78% across three diverse benchmark datasets. Our evaluation demonstrates that WIBA allows the comprehensive understanding of What Is Being Argued in large corpora across diverse contexts, which is of core interest to many applications in linguistics, communication, and social and computer science. To facilitate accessibility to the advancements outlined in this work, we release WIBA as a free open access platform (wiba.dev).