Learning to Fuse Sentences with Transformers for Summarization

TL;DR

This paper investigates how Transformers can be improved to better fuse sentences for summarization by modeling points of correspondence, leading to more accurate and succinct summaries.

Contribution

It introduces novel algorithms that leverage sentence correspondence points to enhance Transformers' sentence fusion capabilities in summarization.

Findings

Modeling points of correspondence improves fusion quality

Transformers with proposed algorithms produce more accurate summaries

Design choices significantly affect fusion performance

Abstract

The ability to fuse sentences is highly attractive for summarization systems because it is an essential step to produce succinct abstracts. However, to date, summarizers can fail on fusing sentences. They tend to produce few summary sentences by fusion or generate incorrect fusions that lead the summary to fail to retain the original meaning. In this paper, we explore the ability of Transformers to fuse sentences and propose novel algorithms to enhance their ability to perform sentence fusion by leveraging the knowledge of points of correspondence between sentences. Through extensive experiments, we investigate the effects of different design choices on Transformer's performance. Our findings highlight the importance of modeling points of correspondence between sentences for effective sentence fusion.