Topic-Aware Encoding for Extractive Summarization

TL;DR

This paper introduces a topic-aware encoding method for extractive summarization that incorporates central topic information into sentence representations, improving the quality of summaries especially for long documents.

Contribution

It proposes a novel model combining syntactic and topic-level information with neural topic modeling to enhance extractive summarization performance.

Findings

Outperforms state-of-the-art models on three datasets

Effectively captures central topics for better summary relevance

Improves summarization quality for lengthy documents

Abstract

Document summarization provides an instrument for faster understanding the collection of text documents and has several real-life applications. With the growth of online text data, numerous summarization models have been proposed recently. The Sequence-to-Sequence (Seq2Seq) based neural summarization model is the most widely used in the summarization field due to its high performance. This is because semantic information and structure information in the text is adequately considered when encoding. However, the existing extractive summarization models pay little attention to and use the central topic information to assist the generation of summaries, which leads to models not ensuring the generated summary under the primary topic. A lengthy document can span several topics, and a single summary cannot do justice to all the topics. Therefore, the key to generating a high-quality summary is determining the central topic and building a summary based on it, especially for a long document. We propose a topic-aware encoding for document summarization to deal with this issue. This model effectively combines syntactic-level and topic-level information to build a comprehensive sentence representation. Specifically, a neural topic model is added in the neural-based sentence-level representation learning to adequately consider the central topic information for capturing the critical content in the original document. The experimental results on three public datasets show that our model outperforms the state-of-the-art models.