A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents

TL;DR

This paper introduces a novel hierarchical, discourse-aware neural model designed for abstractive summarization of long documents like research papers, demonstrating significant improvements over existing models on scientific datasets.

Contribution

It presents the first discourse-structured hierarchical encoder and attentive decoder tailored for long document summarization, advancing the capabilities of neural summarization models.

Findings

Outperforms state-of-the-art models on scientific paper datasets

Effectively models discourse structure for better summaries

Significant improvement in long document summarization quality

Abstract

Neural abstractive summarization models have led to promising results in summarizing relatively short documents. We propose the first model for abstractive summarization of single, longer-form documents (e.g., research papers). Our approach consists of a new hierarchical encoder that models the discourse structure of a document, and an attentive discourse-aware decoder to generate the summary. Empirical results on two large-scale datasets of scientific papers show that our model significantly outperforms state-of-the-art models.