A Deep Generative Framework for Paraphrase Generation

TL;DR

This paper introduces a deep generative model combining VAE and sequence-to-sequence techniques, conditioned on input sentences, to generate multiple high-quality paraphrases, outperforming existing methods on benchmark datasets.

Contribution

It presents a simple, modular model that effectively generates multiple paraphrases for a given sentence, improving over state-of-the-art approaches.

Findings

Significant performance improvement over existing methods.

Generated paraphrases are grammatically correct and relevant.

Established a new baseline on a question paraphrase dataset.

Abstract

Paraphrase generation is an important problem in NLP, especially in question answering, information retrieval, information extraction, conversation systems, to name a few. In this paper, we address the problem of generating paraphrases automatically. Our proposed method is based on a combination of deep generative models (VAE) with sequence-to-sequence models (LSTM) to generate paraphrases, given an input sentence. Traditional VAEs when combined with recurrent neural networks can generate free text but they are not suitable for paraphrase generation for a given sentence. We address this problem by conditioning the both, encoder and decoder sides of VAE, on the original sentence, so that it can generate the given sentence's paraphrases. Unlike most existing models, our model is simple, modular and can generate multiple paraphrases, for a given sentence. Quantitative evaluation of the proposed method on a benchmark paraphrase dataset demonstrates its efficacy, and its performance improvement over the state-of-the-art methods by a significant margin, whereas qualitative human evaluation indicate that the generated paraphrases are well-formed, grammatically correct, and are relevant to the input sentence. Furthermore, we evaluate our method on a newly released question paraphrase dataset, and establish a new baseline for future research.