Q-learning with Language Model for Edit-based Unsupervised Summarization

TL;DR

This paper introduces a novel unsupervised summarization method using Q-learning to train an edit-based model that generates summaries without requiring paired training data, achieving competitive results.

Contribution

The paper presents a new Q-learning based approach with an edit-based framework for unsupervised summarization, integrating reinforcement learning techniques into the process.

Findings

EALM achieved competitive performance without paired data.

The method leverages reinforcement learning for training the summarization agent.

Qualitative analysis offers insights for future unsupervised summarizer development.

Abstract

Unsupervised methods are promising for abstractive text summarization in that the parallel corpora is not required. However, their performance is still far from being satisfied, therefore research on promising solutions is on-going. In this paper, we propose a new approach based on Q-learning with an edit-based summarization. The method combines two key modules to form an Editorial Agent and Language Model converter (EALM). The agent predicts edit actions (e.t., delete, keep, and replace), and then the LM converter deterministically generates a summary on the basis of the action signals. Q-learning is leveraged to train the agent to produce proper edit actions. Experimental results show that EALM delivered competitive performance compared with the previous encoder-decoder-based methods, even with truly zero paired data (i.e., no validation set). Defining the task as Q-learning enables us not only to develop a competitive method but also to make the latest techniques in reinforcement learning available for unsupervised summarization. We also conduct qualitative analysis, providing insights into future study on unsupervised summarizers.