D-PAGE: Diverse Paraphrase Generation

TL;DR

This paper introduces D-PAGE, a neural model extension for generating highly diverse paraphrases, significantly outperforming existing methods in diversity metrics while maintaining quality.

Contribution

D-PAGE is a simple yet effective extension of NMT models that explicitly supports diverse paraphrase generation through implicit rewriting patterns.

Findings

D-PAGE produces at least ten times more diverse outputs than baselines.

The model maintains paraphrase quality while increasing diversity.

Extensive experiments reveal properties of the model related to diversity.

Abstract

In this paper, we investigate the diversity aspect of paraphrase generation. Prior deep learning models employ either decoding methods or add random input noise for varying outputs. We propose a simple method Diverse Paraphrase Generation (D-PAGE), which extends neural machine translation (NMT) models to support the generation of diverse paraphrases with implicit rewriting patterns. Our experimental results on two real-world benchmark datasets demonstrate that our model generates at least one order of magnitude more diverse outputs than the baselines in terms of a new evaluation metric Jeffrey's Divergence. We have also conducted extensive experiments to understand various properties of our model with a focus on diversity.