Heterogeneous Graph Neural Networks for Extractive Document Summarization

TL;DR

This paper introduces HeterSumGraph, a heterogeneous graph neural network that incorporates multiple node types to better model cross-sentence and cross-document relations for extractive summarization.

Contribution

It is the first to integrate different node types into graph neural networks for extractive summarization and extends the model from single to multi-document settings.

Findings

Enriched cross-sentence relations improve summarization quality.

HeterSumGraph outperforms baseline models on benchmark datasets.

Qualitative analysis shows benefits of multi-type nodes.

Abstract

As a crucial step in extractive document summarization, learning cross-sentence relations has been explored by a plethora of approaches. An intuitive way is to put them in the graph-based neural network, which has a more complex structure for capturing inter-sentence relationships. In this paper, we present a heterogeneous graph-based neural network for extractive summarization (HeterSumGraph), which contains semantic nodes of different granularity levels apart from sentences. These additional nodes act as the intermediary between sentences and enrich the cross-sentence relations. Besides, our graph structure is flexible in natural extension from a single-document setting to multi-document via introducing document nodes. To our knowledge, we are the first one to introduce different types of nodes into graph-based neural networks for extractive document summarization and perform a comprehensive qualitative analysis to investigate their benefits. The code will be released on Github