Are Neural Language Models Good Plagiarists? A Benchmark for Neural Paraphrase Detection

TL;DR

This paper introduces a benchmark dataset of paraphrased articles generated by modern language models to evaluate and improve paraphrase detection systems, addressing academic integrity concerns.

Contribution

It provides a large aligned dataset of original and paraphrased texts, analyzes their structure, and evaluates state-of-the-art detection systems, facilitating future research.

Findings

Benchmark dataset of paraphrased articles created

State-of-the-art systems evaluated on the dataset

Findings publicly available for research use

Abstract

The rise of language models such as BERT allows for high-quality text paraphrasing. This is a problem to academic integrity, as it is difficult to differentiate between original and machine-generated content. We propose a benchmark consisting of paraphrased articles using recent language models relying on the Transformer architecture. Our contribution fosters future research of paraphrase detection systems as it offers a large collection of aligned original and paraphrased documents, a study regarding its structure, classification experiments with state-of-the-art systems, and we make our findings publicly available.