Detecting Winning Arguments with Large Language Models and Persuasion Strategies

TL;DR

This paper explores how large language models can detect persuasive strategies in argumentative texts, improving persuasiveness prediction by guiding reasoning with strategy-aware prompts and analyzing topic influences.

Contribution

It introduces a multi-strategy persuasion scoring method using LLMs, and provides a topic-annotated dataset to advance research in argument analysis.

Findings

Strategy-guided reasoning enhances persuasiveness prediction.

Topic organization reveals performance variations across discussion themes.

Public dataset release supports future argumentation research.

Abstract

Detecting persuasion in argumentative text is a challenging task with important implications for understanding human communication. This work investigates the role of persuasion strategies - such as Attack on reputation, Distraction, and Manipulative wording - in determining the persuasiveness of a text. We conduct experiments on three annotated argument datasets: Winning Arguments (built from the Change My View subreddit), Anthropic/Persuasion, and Persuasion for Good. Our approach leverages large language models (LLMs) with a Multi-Strategy Persuasion Scoring approach that guides reasoning over six persuasion strategies. Results show that strategy-guided reasoning improves the prediction of persuasiveness. To better understand the influence of content, we organize the Winning Argument dataset into broad discussion topics and analyze performance across them. We publicly release this topic-annotated version of the dataset to facilitate future research. Overall, our methodology demonstrates the value of structured, strategy-aware prompting for enhancing interpretability and robustness in argument quality assessment.