Hierarchical Sketch Induction for Paraphrase Generation

TL;DR

This paper introduces HRQ-VAE, a hierarchical generative model that encodes syntactic sketches for improved paraphrase generation, demonstrating higher quality outputs through extensive experiments and human evaluation.

Contribution

The paper presents HRQ-VAE, a novel hierarchical VAE that learns to encode syntactic structures as discrete, iterative refinements for better paraphrase generation.

Findings

HRQ-VAE effectively encodes syntactic sketches as hierarchical latent variables.

The model generates paraphrases with higher quality than previous methods.

Human evaluation confirms the superiority of HRQ-VAE in paraphrase quality.

Abstract

We propose a generative model of paraphrase generation, that encourages syntactic diversity by conditioning on an explicit syntactic sketch. We introduce Hierarchical Refinement Quantized Variational Autoencoders (HRQ-VAE), a method for learning decompositions of dense encodings as a sequence of discrete latent variables that make iterative refinements of increasing granularity. This hierarchy of codes is learned through end-to-end training, and represents fine-to-coarse grained information about the input. We use HRQ-VAE to encode the syntactic form of an input sentence as a path through the hierarchy, allowing us to more easily predict syntactic sketches at test time. Extensive experiments, including a human evaluation, confirm that HRQ-VAE learns a hierarchical representation of the input space, and generates paraphrases of higher quality than previous systems.