A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss

TL;DR

This paper introduces a unified summarization model that combines extractive and abstractive approaches with an inconsistency loss to improve readability and informativeness, achieving state-of-the-art results.

Contribution

The paper presents a novel unified model with an inconsistency loss that effectively integrates extractive and abstractive summarization techniques.

Findings

Achieves state-of-the-art ROUGE scores on CNN/Daily Mail dataset.

Produces more readable and informative summaries according to human evaluation.

Effectively balances extractive and abstractive strengths through end-to-end training.

Abstract

We propose a unified model combining the strength of extractive and abstractive summarization. On the one hand, a simple extractive model can obtain sentence-level attention with high ROUGE scores but less readable. On the other hand, a more complicated abstractive model can obtain word-level dynamic attention to generate a more readable paragraph. In our model, sentence-level attention is used to modulate the word-level attention such that words in less attended sentences are less likely to be generated. Moreover, a novel inconsistency loss function is introduced to penalize the inconsistency between two levels of attentions. By end-to-end training our model with the inconsistency loss and original losses of extractive and abstractive models, we achieve state-of-the-art ROUGE scores while being the most informative and readable summarization on the CNN/Daily Mail dataset in a solid human evaluation.