Unsupervised Abstractive Opinion Summarization by Generating Sentences with Tree-Structured Topic Guidance

TL;DR

This paper introduces an unsupervised abstractive summarization approach that uses a tree-structured Gaussian mixture model to generate topic-guided sentences, improving summary informativeness and content coverage.

Contribution

It proposes a novel recursive Gaussian mixture prior for variational autoencoders, enabling tree-structured topic guidance in sentence generation for opinion summarization.

Findings

Generated summaries are more informative and comprehensive.

The model's Gaussian variance correlates with sentence granularity.

Outperforms recent unsupervised summarization models.

Abstract

This paper presents a novel unsupervised abstractive summarization method for opinionated texts. While the basic variational autoencoder-based models assume a unimodal Gaussian prior for the latent code of sentences, we alternate it with a recursive Gaussian mixture, where each mixture component corresponds to the latent code of a topic sentence and is mixed by a tree-structured topic distribution. By decoding each Gaussian component, we generate sentences with tree-structured topic guidance, where the root sentence conveys generic content, and the leaf sentences describe specific topics. Experimental results demonstrate that the generated topic sentences are appropriate as a summary of opinionated texts, which are more informative and cover more input contents than those generated by the recent unsupervised summarization model (Bra\v{z}inskas et al., 2020). Furthermore, we demonstrate that the variance of latent Gaussians represents the granularity of sentences, analogous to Gaussian word embedding (Vilnis and McCallum, 2015).