ParaNMT-50M: Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations

TL;DR

ParaNMT-50M is a large-scale dataset of over 50 million paraphrase pairs created via neural machine translation, aimed at improving paraphrastic sentence embeddings and natural language understanding.

Contribution

It introduces a massive paraphrase dataset generated through machine translation, enabling significant advancements in sentence embeddings and paraphrase generation.

Findings

Embeddings trained on ParaNMT-50M outperform supervised systems on SemEval STS tasks.

The dataset effectively enhances paraphrase generation capabilities.

Demonstrates the utility of large-scale paraphrase data for NLP applications.

Abstract

We describe PARANMT-50M, a dataset of more than 50 million English-English sentential paraphrase pairs. We generated the pairs automatically by using neural machine translation to translate the non-English side of a large parallel corpus, following Wieting et al. (2017). Our hope is that ParaNMT-50M can be a valuable resource for paraphrase generation and can provide a rich source of semantic knowledge to improve downstream natural language understanding tasks. To show its utility, we use ParaNMT-50M to train paraphrastic sentence embeddings that outperform all supervised systems on every SemEval semantic textual similarity competition, in addition to showing how it can be used for paraphrase generation.