Adversarial Example Generation with Syntactically Controlled Paraphrase Networks

TL;DR

This paper introduces syntactically controlled paraphrase networks (SCPNs) that generate paraphrases with specific syntactic structures, used to create adversarial examples and improve model robustness.

Contribution

The paper presents SCPNs, a novel neural model for syntax-controlled paraphrasing, trained via large-scale backtranslation data and parsing, enhancing adversarial example generation and model robustness.

Findings

SCPNs produce syntactically accurate paraphrases without quality loss.

They generate effective adversarial examples that fool pretrained models.

Using SCPN-generated data improves model robustness to syntactic variations.

Abstract

We propose syntactically controlled paraphrase networks (SCPNs) and use them to generate adversarial examples. Given a sentence and a target syntactic form (e.g., a constituency parse), SCPNs are trained to produce a paraphrase of the sentence with the desired syntax. We show it is possible to create training data for this task by first doing backtranslation at a very large scale, and then using a parser to label the syntactic transformations that naturally occur during this process. Such data allows us to train a neural encoder-decoder model with extra inputs to specify the target syntax. A combination of automated and human evaluations show that SCPNs generate paraphrases that follow their target specifications without decreasing paraphrase quality when compared to baseline (uncontrolled) paraphrase systems. Furthermore, they are more capable of generating syntactically adversarial examples that both (1) "fool" pretrained models and (2) improve the robustness of these models to syntactic variation when used to augment their training data.