How to Train Text Summarization Model with Weak Supervisions

TL;DR

This paper introduces a novel training method for text summarization that utilizes weak supervision signals derived from decomposing complex objectives, enabling effective end-to-end training without labeled data.

Contribution

It proposes a new approach to generate supervision signals from complex tasks, facilitating training without explicit labels, demonstrated on topic-based summarization.

Findings

Achieves strong performance on CNN and DailyMail datasets.

Enables end-to-end training without labeled data.

Effectively leverages rich supervision signals for summarization.

Abstract

Currently, machine learning techniques have seen significant success across various applications. Most of these techniques rely on supervision from human-generated labels or a mixture of noisy and imprecise labels from multiple sources. However, for certain complex tasks, even noisy or inexact labels are unavailable due to the intricacy of the objectives. To tackle this issue, we propose a method that breaks down the complex objective into simpler tasks and generates supervision signals for each one. We then integrate these supervision signals into a manageable form, resulting in a straightforward learning procedure. As a case study, we demonstrate a system used for topic-based summarization. This system leverages rich supervision signals to promote both summarization and topic relevance. Remarkably, we can train the model end-to-end without any labels. Experimental results indicate that our approach performs exceptionally well on the CNN and DailyMail datasets.