Quantitative Argument Summarization and Beyond: Cross-Domain Key Point Analysis

TL;DR

This paper introduces an automatic method for extracting key points from argumentation data, enabling quantitative analysis across multiple domains, and demonstrates its effectiveness compared to human performance and in diverse datasets.

Contribution

It presents a novel automatic key point extraction method and shows its applicability beyond argumentation, including surveys and reviews, with improved matching performance.

Findings

Automatic key point extraction achieves human-level performance.

Method generalizes well to surveys and reviews.

Significant improvement in argument-to-key point matching accuracy.

Abstract

When summarizing a collection of views, arguments or opinions on some topic, it is often desirable not only to extract the most salient points, but also to quantify their prevalence. Work on multi-document summarization has traditionally focused on creating textual summaries, which lack this quantitative aspect. Recent work has proposed to summarize arguments by mapping them to a small set of expert-generated key points, where the salience of each key point corresponds to the number of its matching arguments. The current work advances key point analysis in two important respects: first, we develop a method for automatic extraction of key points, which enables fully automatic analysis, and is shown to achieve performance comparable to a human expert. Second, we demonstrate that the applicability of key point analysis goes well beyond argumentation data. Using models trained on publicly available argumentation datasets, we achieve promising results in two additional domains: municipal surveys and user reviews. An additional contribution is an in-depth evaluation of argument-to-key point matching models, where we substantially outperform previous results.