Exploring Multitask Learning for Low-Resource AbstractiveSummarization

TL;DR

This study investigates how multitask learning with related tasks can improve low-resource abstractive summarization, demonstrating that certain auxiliary tasks significantly enhance summarization quality without extra summarization data.

Contribution

The paper introduces a comprehensive analysis of multitask learning for low-resource abstractive summarization, identifying effective auxiliary tasks and optimal combinations to boost performance.

Findings

Multitask learning improves summarization performance over single-task models.

Paraphrase detection consistently benefits abstractive summarization.

Certain task combinations outperform individual tasks in low-resource settings.

Abstract

This paper explores the effect of using multitask learning for abstractive summarization in the context of small training corpora. In particular, we incorporate four different tasks (extractive summarization, language modeling, concept detection, and paraphrase detection) both individually and in combination, with the goal of enhancing the target task of abstractive summarization via multitask learning. We show that for many task combinations, a model trained in a multitask setting outperforms a model trained only for abstractive summarization, with no additional summarization data introduced. Additionally, we do a comprehensive search and find that certain tasks (e.g. paraphrase detection) consistently benefit abstractive summarization, not only when combined with other tasks but also when using different architectures and training corpora.