Improving Large-scale Paraphrase Acquisition and Generation

TL;DR

This paper introduces the MultiPIT corpus with high-quality annotations for paraphrase identification and generation, leading to state-of-the-art results and more diverse paraphrases in large-scale Twitter data.

Contribution

It presents a new multi-topic Twitter paraphrase dataset with improved annotations and task-specific definitions, enhancing paraphrase identification and generation models.

Findings

State-of-the-art 84.2 F1 in paraphrase identification

Models trained on MultiPIT produce more diverse, high-quality paraphrases

New dataset improves both identification and generation tasks

Abstract

This paper addresses the quality issues in existing Twitter-based paraphrase datasets, and discusses the necessity of using two separate definitions of paraphrase for identification and generation tasks. We present a new Multi-Topic Paraphrase in Twitter (MultiPIT) corpus that consists of a total of 130k sentence pairs with crowdsoursing (MultiPIT_crowd) and expert (MultiPIT_expert) annotations using two different paraphrase definitions for paraphrase identification, in addition to a multi-reference test set (MultiPIT_NMR) and a large automatically constructed training set (MultiPIT_Auto) for paraphrase generation. With improved data annotation quality and task-specific paraphrase definition, the best pre-trained language model fine-tuned on our dataset achieves the state-of-the-art performance of 84.2 F1 for automatic paraphrase identification. Furthermore, our empirical results also demonstrate that the paraphrase generation models trained on MultiPIT_Auto generate more diverse and high-quality paraphrases compared to their counterparts fine-tuned on other corpora such as Quora, MSCOCO, and ParaNMT.