Discourse-Aware Neural Extractive Text Summarization

TL;DR

DiscoBert is a discourse-aware neural extractive summarization model that uses discourse units and graph-based encoding to improve summary quality by capturing long-range dependencies and reducing redundancy.

Contribution

The paper introduces DiscoBert, a novel model that extracts discourse units and employs graph convolutional networks to better capture document structure for summarization.

Findings

Outperforms state-of-the-art models on benchmark datasets

Effectively captures long-range dependencies in documents

Reduces redundancy and uninformative phrases in summaries

Abstract

Recently BERT has been adopted for document encoding in state-of-the-art text summarization models. However, sentence-based extractive models often result in redundant or uninformative phrases in the extracted summaries. Also, long-range dependencies throughout a document are not well captured by BERT, which is pre-trained on sentence pairs instead of documents. To address these issues, we present a discourse-aware neural summarization model - DiscoBert. DiscoBert extracts sub-sentential discourse units (instead of sentences) as candidates for extractive selection on a finer granularity. To capture the long-range dependencies among discourse units, structural discourse graphs are constructed based on RST trees and coreference mentions, encoded with Graph Convolutional Networks. Experiments show that the proposed model outperforms state-of-the-art methods by a significant margin on popular summarization benchmarks compared to other BERT-base models.