Demoting the Lead Bias in News Summarization via Alternating Adversarial Learning

TL;DR

This paper presents a novel adversarial learning approach to reduce lead bias in news summarization models, enhancing their ability to generalize across diverse datasets without sacrificing in-distribution performance.

Contribution

It introduces a new technique that effectively demotes lead bias in neural extractive summarizers using alternating adversarial learning, improving out-of-distribution generalization.

Findings

Reduces lead bias in summarization models

Improves performance on out-of-distribution data

Maintains in-distribution summarization quality

Abstract

In news articles the lead bias is a common phenomenon that usually dominates the learning signals for neural extractive summarizers, severely limiting their performance on data with different or even no bias. In this paper, we introduce a novel technique to demote lead bias and make the summarizer focus more on the content semantics. Experiments on two news corpora with different degrees of lead bias show that our method can effectively demote the model's learned lead bias and improve its generality on out-of-distribution data, with little to no performance loss on in-distribution data.