Context-Aware Hierarchical Merging for Long Document Summarization

TL;DR

This paper enhances hierarchical merging for long document summarization by integrating source context to reduce hallucinations and improve factual accuracy, demonstrating superior results on legal and narrative datasets.

Contribution

It introduces context augmentation techniques—replacement, refinement, and alignment—to improve the factual reliability of hierarchical merging in long document summarization.

Findings

Contextual augmentation outperforms baseline methods.

Refinement works best with extractive summarization.

Methods reduce hallucinations and improve factual accuracy.

Abstract

Hierarchical Merging is a technique commonly used to summarize very long texts ($>$100K tokens) by breaking down the input into smaller sections, summarizing those sections individually, and then merging or combining those summaries into a final coherent summary. Although it helps address the limitations of large language models (LLMs) with fixed input length constraints, the recursive merging process can amplify LLM hallucinations, increasing the risk of factual inaccuracies. In this paper, we seek to mitigate hallucinations by enriching hierarchical merging with context from the source document. Specifically, we propose different approaches to contextual augmentation ranging from \emph{replacing} intermediate summaries with relevant input context, to \emph{refining} them while using the context as supporting evidence, and \emph{aligning} them implicitly (via citations) to the input. Experimental results on datasets representing legal and narrative domains show that contextual augmentation consistently outperforms zero-shot and hierarchical merging baselines for the Llama 3.1 model family. Our analysis further reveals that refinement methods tend to perform best when paired with extractive summarization for identifying relevant input.