Hierarchical Sketch Induction for Paraphrase Generation
Tom Hosking, Hao Tang, Mirella Lapata

TL;DR
This paper introduces HRQ-VAE, a hierarchical generative model that encodes syntactic sketches for improved paraphrase generation, demonstrating higher quality outputs through extensive experiments and human evaluation.
Contribution
The paper presents HRQ-VAE, a novel hierarchical VAE that learns to encode syntactic structures as discrete, iterative refinements for better paraphrase generation.
Findings
HRQ-VAE effectively encodes syntactic sketches as hierarchical latent variables.
The model generates paraphrases with higher quality than previous methods.
Human evaluation confirms the superiority of HRQ-VAE in paraphrase quality.
Abstract
We propose a generative model of paraphrase generation, that encourages syntactic diversity by conditioning on an explicit syntactic sketch. We introduce Hierarchical Refinement Quantized Variational Autoencoders (HRQ-VAE), a method for learning decompositions of dense encodings as a sequence of discrete latent variables that make iterative refinements of increasing granularity. This hierarchy of codes is learned through end-to-end training, and represents fine-to-coarse grained information about the input. We use HRQ-VAE to encode the syntactic form of an input sentence as a path through the hierarchy, allowing us to more easily predict syntactic sketches at test time. Extensive experiments, including a human evaluation, confirm that HRQ-VAE learns a hierarchical representation of the input space, and generates paraphrases of higher quality than previous systems.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Multimodal Machine Learning Applications
