Spotting AI's Touch: Identifying LLM-Paraphrased Spans in Text

TL;DR

This paper introduces PTD, a new framework for detecting AI-paraphrased text spans within full texts, supported by a dedicated dataset, demonstrating effectiveness and generalization across various paraphrasing scenarios.

Contribution

The paper presents a novel paraphrased text span detection method and a new dataset, PASTED, for identifying AI-generated paraphrased segments within texts.

Findings

PTD effectively detects paraphrased spans in full texts.

Models generalize well to different paraphrasing prompts.

Context surrounding paraphrased spans is crucial for detection accuracy.

Abstract

AI-generated text detection has attracted increasing attention as powerful language models approach human-level generation. Limited work is devoted to detecting (partially) AI-paraphrased texts. However, AI paraphrasing is commonly employed in various application scenarios for text refinement and diversity. To this end, we propose a novel detection framework, paraphrased text span detection (PTD), aiming to identify paraphrased text spans within a text. Different from text-level detection, PTD takes in the full text and assigns each of the sentences with a score indicating the paraphrasing degree. We construct a dedicated dataset, PASTED, for paraphrased text span detection. Both in-distribution and out-of-distribution results demonstrate the effectiveness of PTD models in identifying AI-paraphrased text spans. Statistical and model analysis explains the crucial role of the surrounding context of the paraphrased text spans. Extensive experiments show that PTD models can generalize to versatile paraphrasing prompts and multiple paraphrased text spans. We release our resources at https://github.com/Linzwcs/PASTED.