AlignSum: Data Pyramid Hierarchical Fine-tuning for Aligning with Human Summarization Preference

TL;DR

AlignSum introduces a hierarchical fine-tuning framework using a data pyramid and Gaussian resampling to better align pre-trained language models with human summarization preferences, significantly improving human evaluation scores.

Contribution

This paper presents a novel data pyramid and a two-stage hierarchical fine-tuning method to improve the alignment of PLMs with human summarization preferences, addressing dataset quality issues.

Findings

PLMs with AlignSum outperform GPT-3 in human evaluations.

Hierarchical fine-tuning improves alignment with human preferences.

Data pyramid and Gaussian resampling enhance dataset quality.

Abstract

Text summarization tasks commonly employ Pre-trained Language Models (PLMs) to fit diverse standard datasets. While these PLMs excel in automatic evaluations, they frequently underperform in human evaluations, indicating a deviation between their generated summaries and human summarization preferences. This discrepancy is likely due to the low quality of fine-tuning datasets and the limited availability of high-quality human-annotated data that reflect true human preference. To address this challenge, we introduce a novel human summarization preference alignment framework AlignSum. This framework consists of three parts: Firstly, we construct a Data Pymarid with extractive, abstractive, and human-annotated summary data. Secondly, we conduct the Gaussian Resampling to remove summaries with extreme lengths. Finally, we implement the two-stage hierarchical fine-tuning with Data Pymarid after Gaussian Resampling. We apply AlignSum to PLMs on the human-annotated CNN/DailyMail and BBC XSum datasets. Experiments show that with AlignSum, PLMs like BART-Large surpass 175B GPT-3 in both automatic and human evaluations. This demonstrates that AlignSum significantly enhances the alignment of language models with human summarization preferences.