Convex Aggregation for Opinion Summarization

TL;DR

This paper investigates how simple averaging of latent vectors in autoencoder-based opinion summarization can lead to overly generic summaries due to vector degeneration, and proposes a new framework called Coop to improve summary quality.

Contribution

It introduces Coop, a novel framework that optimizes latent vector aggregation by using input-output word overlap, addressing the degeneration issue and achieving state-of-the-art results.

Findings

Coop alleviates summary vector degeneration.

Achieves new state-of-the-art on opinion summarization benchmarks.

Addresses the impact of latent space aggregation on summary quality.

Abstract

Recent advances in text autoencoders have significantly improved the quality of the latent space, which enables models to generate grammatical and consistent text from aggregated latent vectors. As a successful application of this property, unsupervised opinion summarization models generate a summary by decoding the aggregated latent vectors of inputs. More specifically, they perform the aggregation via simple average. However, little is known about how the vector aggregation step affects the generation quality. In this study, we revisit the commonly used simple average approach by examining the latent space and generated summaries. We found that text autoencoders tend to generate overly generic summaries from simply averaged latent vectors due to an unexpected $L_2$-norm shrinkage in the aggregated latent vectors, which we refer to as summary vector degeneration. To overcome this issue, we develop a framework Coop, which searches input combinations for the latent vector aggregation using input-output word overlap. Experimental results show that Coop successfully alleviates the summary vector degeneration issue and establishes new state-of-the-art performance on two opinion summarization benchmarks. Code is available at \url{https://github.com/megagonlabs/coop}.