Sequence Level Contrastive Learning for Text Summarization

TL;DR

This paper introduces a contrastive learning approach for supervised abstractive text summarization, enhancing model faithfulness and similarity between documents and summaries, and improving performance over existing models.

Contribution

It proposes a novel contrastive learning framework that treats documents, gold summaries, and generated summaries as different views to improve summarization quality.

Findings

Outperforms BART on three datasets.

Achieves higher faithfulness ratings in human evaluations.

Enhances similarity between related text views.

Abstract

Contrastive learning models have achieved great success in unsupervised visual representation learning, which maximize the similarities between feature representations of different views of the same image, while minimize the similarities between feature representations of views of different images. In text summarization, the output summary is a shorter form of the input document and they have similar meanings. In this paper, we propose a contrastive learning model for supervised abstractive text summarization, where we view a document, its gold summary and its model generated summaries as different views of the same mean representation and maximize the similarities between them during training. We improve over a strong sequence-to-sequence text generation model (i.e., BART) on three different summarization datasets. Human evaluation also shows that our model achieves better faithfulness ratings compared to its counterpart without contrastive objectives.