ReDecode Framework for Iterative Improvement in Paraphrase Generation

TL;DR

This paper introduces an iterative refinement framework for paraphrase generation using multiple decoders within a VAE, significantly improving quality and coherence over existing models.

Contribution

It proposes a novel re-decoding approach with multiple decoders for iterative improvement in paraphrase generation, addressing limitations of current sequence models.

Findings

Over 9% and 28% absolute increase in METEOR scores on Quora and MSCOCO datasets.

Qualitative examples show improved paraphrase quality and error correction.

Significant enhancement over state-of-the-art results.

Abstract

Generating paraphrases, that is, different variations of a sentence conveying the same meaning, is an important yet challenging task in NLP. Automatically generating paraphrases has its utility in many NLP tasks like question answering, information retrieval, conversational systems to name a few. In this paper, we introduce iterative refinement of generated paraphrases within VAE based generation framework. Current sequence generation models lack the capability to (1) make improvements once the sentence is generated; (2) rectify errors made while decoding. We propose a technique to iteratively refine the output using multiple decoders, each one attending on the output sentence generated by the previous decoder. We improve current state of the art results significantly - with over 9% and 28% absolute increase in METEOR scores on Quora question pairs and MSCOCO datasets respectively. We also show qualitatively through examples that our re-decoding approach generates better paraphrases compared to a single decoder by rectifying errors and making improvements in paraphrase structure, inducing variations and introducing new but semantically coherent information.