Inverse Reinforcement Learning for Text Summarization

TL;DR

This paper presents an inverse reinforcement learning approach for training abstractive text summarization models, which better imitates human summarization and improves multiple evaluation metrics across diverse datasets.

Contribution

It introduces IRL for summarization, estimating reward functions from human-like behaviors, and demonstrates its effectiveness over traditional methods like MLE and RL.

Findings

IRL model outperforms MLE and RL baselines

Summaries have higher ROUGE, coverage, and novelty scores

Model achieves better factuality and human evaluation results

Abstract

We introduce inverse reinforcement learning (IRL) as an effective paradigm for training abstractive summarization models, imitating human summarization behaviors. Our IRL model estimates the reward function using a suite of important sub-rewards for summarization and concurrently optimizes the policy network. Experimental results across datasets in different domains (CNN/DailyMail and WikiHow) and various model sizes (BART-base and BART-large) demonstrate the superiority of our proposed IRL model for summarization over MLE and RL baselines. The resulting summaries exhibit greater similarity to human-crafted gold references, outperforming MLE and RL baselines on metrics such as ROUGE, coverage, novelty, compression ratio, factuality, and human evaluations.