HERA: Improving Long Document Summarization using Large Language Models with Context Packaging and Reordering

TL;DR

HERA is a framework that enhances long document summarization by segmenting, retrieving, and reordering content to improve understanding and relevance in large language models without extra fine-tuning.

Contribution

HERA introduces a novel method of semantic segmentation, event-based retrieval, and reordering to improve long document summarization with LLMs.

Findings

HERA outperforms baseline models in ROUGE, BERTScore, and faithfulness.

It does not require additional fine-tuning or resources.

Effective in two long document summarization datasets.

Abstract

Despite the rapid growth of context length of large language models (LLMs) , LLMs still perform poorly in long document summarization. An important reason for this is that relevant information about an event is scattered throughout long documents, and the messy narrative order impairs the accurate understanding and utilization of LLMs for long documents. To address these issues, we propose a novel summary generation framework, called HERA. Specifically, we first segment a long document by its semantic structure and retrieve text segments about the same event, and finally reorder them to form the input context. We evaluate our approach on two long document summarization datasets. The experimental results show that HERA outperforms foundation models in ROUGE, BERTScore and faithfulness metrics, while HERA does not require additional fine-tuning and resources.