Abstractive and mixed summarization for long-single documents

TL;DR

This paper explores abstractive and mixed summarization techniques for long scientific documents, demonstrating that hierarchical encoder models outperform others in capturing document structure.

Contribution

It introduces a comparative analysis of six models on scientific papers, highlighting the effectiveness of hierarchical encoders for long document summarization.

Findings

Hierarchical encoder models outperform other architectures.

Transformer-based models with reinforcement learning show improved results.

Long scientific papers can be effectively summarized using these models.

Abstract

The lack of diversity in the datasets available for automatic summarization of documents has meant that the vast majority of neural models for automatic summarization have been trained with news articles. These datasets are relatively small, with an average size of about 600 words, and the models trained with such data sets see their performance limited to short documents. In order to surmount this problem, this paper uses scientific papers as the dataset on which different models are trained. These models have been chosen based on their performance on the CNN/Daily Mail data set, so that the highest ranked model of each architectural variant is selected. In this work, six different models are compared, two with an RNN architecture, one with a CNN architecture, two with a Transformer architecture and one with a Transformer architecture combined with reinforcement learning. The results from this work show that those models that use a hierarchical encoder to model the structure of the document has a better performance than the rest.