From Key Points to Key Point Hierarchy: Structured and Expressive Opinion Summarization

TL;DR

This paper introduces the task of organizing key points from textual comments into hierarchical structures to better understand related ideas at different levels of granularity, using a new dataset and improved prediction methods.

Contribution

It proposes the novel task of key point hierarchy construction, creates the ThinkP benchmark dataset, and develops advanced methods for predicting hierarchical relations among key points.

Findings

Significant performance improvements over baselines using directional distributional similarity.

Development of the ThinkP dataset for key point hierarchies in reviews.

Effective methods for inferring hierarchies from pairwise key point relations.

Abstract

Key Point Analysis (KPA) has been recently proposed for deriving fine-grained insights from collections of textual comments. KPA extracts the main points in the data as a list of concise sentences or phrases, termed key points, and quantifies their prevalence. While key points are more expressive than word clouds and key phrases, making sense of a long, flat list of key points, which often express related ideas in varying levels of granularity, may still be challenging. To address this limitation of KPA, we introduce the task of organizing a given set of key points into a hierarchy, according to their specificity. Such hierarchies may be viewed as a novel type of Textual Entailment Graph. We develop ThinkP, a high quality benchmark dataset of key point hierarchies for business and product reviews, obtained by consolidating multiple annotations. We compare different methods for predicting pairwise relations between key points, and for inferring a hierarchy from these pairwise predictions. In particular, for the task of computing pairwise key point relations, we achieve significant gains over existing strong baselines by applying directional distributional similarity methods to a novel distributional representation of key points, and further boost performance via weak supervision.