A Reinforced Topic-Aware Convolutional Sequence-to-Sequence Model for Abstractive Text Summarization

TL;DR

This paper introduces a novel deep learning model that integrates topic information into a convolutional sequence-to-sequence framework, optimized with reinforcement learning to improve the quality of abstractive text summaries.

Contribution

It presents a new topic-aware ConvS2S model combined with SCST training, enhancing coherence, diversity, and ROUGE scores in abstractive summarization tasks.

Findings

Outperforms state-of-the-art methods on multiple datasets

Improves summary coherence and informativeness

Effectively optimizes ROUGE scores using reinforcement learning

Abstract

In this paper, we propose a deep learning approach to tackle the automatic summarization tasks by incorporating topic information into the convolutional sequence-to-sequence (ConvS2S) model and using self-critical sequence training (SCST) for optimization. Through jointly attending to topics and word-level alignment, our approach can improve coherence, diversity, and informativeness of generated summaries via a biased probability generation mechanism. On the other hand, reinforcement training, like SCST, directly optimizes the proposed model with respect to the non-differentiable metric ROUGE, which also avoids the exposure bias during inference. We carry out the experimental evaluation with state-of-the-art methods over the Gigaword, DUC-2004, and LCSTS datasets. The empirical results demonstrate the superiority of our proposed method in the abstractive summarization.