The Influence of Data Pre-processing and Post-processing on Long Document Summarization

TL;DR

This paper investigates how data pre-processing and post-processing techniques impact the performance of long document summarization models, highlighting less-explored areas beyond attention mechanism modifications.

Contribution

It introduces and analyzes the effects of specific pre- and post-processing methods on long document summarization models, filling a research gap.

Findings

Pre-processing improves summary quality in certain models.

Post-processing enhances ROUGE scores.

Different methods have varying impacts depending on the model.

Abstract

Long document summarization is an important and hard task in the field of natural language processing. A good performance of the long document summarization reveals the model has a decent understanding of the human language. Currently, most researches focus on how to modify the attention mechanism of the transformer to achieve a higher ROUGE score. The study of data pre-processing and post-processing are relatively few. In this paper, we use two pre-processing methods and a post-processing method and analyze the effect of these methods on various long document summarization models.