ARQUSUMM: Argument-aware Quantitative Summarization of Online Conversations

TL;DR

ARQUSUMM is a novel framework that combines argument structure analysis and quantitative summarization to produce more informative summaries of online conversations, capturing claim-reason relationships and argument strength.

Contribution

It introduces a new argument-aware quantitative summarization task and proposes ARQUSUMM, leveraging LLM few-shot learning and clustering to reveal argument structures within conversations.

Findings

ARQUSUMM outperforms existing models in summarization quality.

Generated summaries better represent argument structures.

The framework achieves high quantification accuracy.

Abstract

Online conversations have become more prevalent on public discussion platforms (e.g. Reddit). With growing controversial topics, it is desirable to summarize not only diverse arguments, but also their rationale and justification. Early studies on text summarization focus on capturing general salient information in source documents, overlooking the argumentative nature of online conversations. Recent research on conversation summarization although considers the argumentative relationship among sentences, fail to explicate deeper argument structure within sentences for summarization. In this paper, we propose a novel task of argument-aware quantitative summarization to reveal the claim-reason structure of arguments in conversations, with quantities measuring argument strength. We further propose ARQUSUMM, a novel framework to address the task. To reveal the underlying argument structure within sentences, ARQUSUMM leverages LLM few-shot learning grounded in the argumentation theory to identify propositions within sentences and their claim-reason relationships. For quantitative summarization, ARQUSUMM employs argument structure-aware clustering algorithms to aggregate arguments and quantify their support. Experiments show that ARQUSUMM outperforms existing conversation and quantitative summarization models and generate summaries representing argument structures that are more helpful to users, of high textual quality and quantification accuracy.