A Novel LLM-based Two-stage Summarization Approach for Long Dialogues

TL;DR

This paper introduces a hierarchical two-stage summarization framework for long dialogues using segmentation, condensation, and fine-tuning of an abstractive model, enabling efficient processing of lengthy texts beyond model input limits.

Contribution

It presents a novel hierarchical approach combining unsupervised segmentation and condensation with fine-tuning, reducing computational costs for long dialogue summarization.

Findings

Effective segmentation and condensation improve summarization quality.

Framework reduces training time and resource requirements.

Applicable to dialogues exceeding standard model input lengths.

Abstract

Long document summarization poses a significant challenge in natural language processing due to input lengths that exceed the capacity of most state-of-the-art pre-trained language models. This study proposes a hierarchical framework that segments and condenses information from long documents, subsequently fine-tuning the processed text with an abstractive summarization model. Unsupervised topic segmentation methods identify semantically appropriate breakpoints. The condensation stage utilizes an unsupervised generation model to generate condensed data, and our current experiments employ ChatGPT(v3.5). The summarization stage fine-tunes the abstractive summarization model on the condensed data to generate the final results. This framework enables long documents to be processed on models even when the document length exceeds the model's maximum input size. The exclusion of the entire document from the summarization model reduces the time and computational resources required for training, making the framework suitable for contexts with constrained local computational resources.