Don't Change Me! User-Controllable Selective Paraphrase Generation

TL;DR

This paper introduces a user-controllable paraphrase generation method that allows explicit tagging of phrases to prevent changes, using a novel data generation technique and fine-tuning of pretrained models, demonstrated in English and Chinese.

Contribution

It presents a new data generation and fine-tuning approach enabling user-controlled paraphrasing with phrase preservation capabilities.

Findings

Effective in English and Chinese

Produces diverse paraphrases

Maintains phrase integrity as specified by user

Abstract

In the paraphrase generation task, source sentences often contain phrases that should not be altered. Which phrases, however, can be context dependent and can vary by application. Our solution to this challenge is to provide the user with explicit tags that can be placed around any arbitrary segment of text to mean "don't change me!" when generating a paraphrase; the model learns to explicitly copy these phrases to the output. The contribution of this work is a novel data generation technique using distant supervision that allows us to start with a pretrained sequence-to-sequence model and fine-tune a paraphrase generator that exhibits this behavior, allowing user-controllable paraphrase generation. Additionally, we modify the loss during fine-tuning to explicitly encourage diversity in model output. Our technique is language agnostic, and we report experiments in English and Chinese.