Improving Factuality of Abstractive Summarization without Sacrificing Summary Quality

TL;DR

EFACTSUM is a novel method that enhances the factual accuracy of abstractive summaries through candidate ranking and contrastive learning, improving factuality metrics without harming summary quality.

Contribution

The paper introduces EFACTSUM, a new candidate generation and ranking approach that improves factual consistency in summarization without sacrificing quality.

Findings

Up to 6 points improvement on FactCC for XSUM

Up to 11 points improvement on FactCC for CNN/DM

No negative impact on summary similarity or abstractiveness

Abstract

Improving factual consistency of abstractive summarization has been a widely studied topic. However, most of the prior works on training factuality-aware models have ignored the negative effect it has on summary quality. We propose EFACTSUM (i.e., Effective Factual Summarization), a candidate summary generation and ranking technique to improve summary factuality without sacrificing summary quality. We show that using a contrastive learning framework with our refined candidate summaries leads to significant gains on both factuality and similarity-based metrics. Specifically, we propose a ranking strategy in which we effectively combine two metrics, thereby preventing any conflict during training. Models trained using our approach show up to 6 points of absolute improvement over the base model with respect to FactCC on XSUM and 11 points on CNN/DM, without negatively affecting either similarity-based metrics or absractiveness.