Structured Neural Summarization

TL;DR

This paper introduces a hybrid sequence-graph neural network framework that enhances summarization by reasoning about long-distance relationships in weakly structured text data, outperforming existing models.

Contribution

It presents a novel hybrid model combining sequence encoders with graph components to improve summarization of long, weakly structured texts.

Findings

Hybrid models outperform pure sequence models.

Hybrid models outperform pure graph models.

Effective reasoning about long-distance relationships.

Abstract

Summarization of long sequences into a concise statement is a core problem in natural language processing, requiring non-trivial understanding of the input. Based on the promising results of graph neural networks on highly structured data, we develop a framework to extend existing sequence encoders with a graph component that can reason about long-distance relationships in weakly structured data such as text. In an extensive evaluation, we show that the resulting hybrid sequence-graph models outperform both pure sequence models as well as pure graph models on a range of summarization tasks.