Soft Layer-Specific Multi-Task Summarization with Entailment and Question Generation

TL;DR

This paper introduces a multi-task learning approach for abstractive summarization that incorporates question generation and entailment tasks, using novel layer-specific sharing mechanisms to improve summary quality and logical consistency.

Contribution

It proposes a new multi-task architecture with high-level layer-specific sharing and soft-sharing, enhancing summarization by integrating entailment and question generation tasks.

Findings

Significant improvements over state-of-the-art on CNN/DailyMail and Gigaword datasets.

Demonstrated better saliency detection and entailment skills in summaries.

Achieved strong transfer performance on DUC-2002 dataset.

Abstract

An accurate abstractive summary of a document should contain all its salient information and should be logically entailed by the input document. We improve these important aspects of abstractive summarization via multi-task learning with the auxiliary tasks of question generation and entailment generation, where the former teaches the summarization model how to look for salient questioning-worthy details, and the latter teaches the model how to rewrite a summary which is a directed-logical subset of the input document. We also propose novel multi-task architectures with high-level (semantic) layer-specific sharing across multiple encoder and decoder layers of the three tasks, as well as soft-sharing mechanisms (and show performance ablations and analysis examples of each contribution). Overall, we achieve statistically significant improvements over the state-of-the-art on both the CNN/DailyMail and Gigaword datasets, as well as on the DUC-2002 transfer setup. We also present several quantitative and qualitative analysis studies of our model's learned saliency and entailment skills.