DART: An AIGT Detector using AMR of Rephrased Text

TL;DR

This paper introduces DART, a novel AIGT detection method that leverages AMR of rephrased text to improve detection of black-box LLM-generated texts in multi-candidate scenarios.

Contribution

DART is the first detector to use AMR of rephrased text for AIGT detection, addressing limitations of probabilistic feature reliance and single-candidate testing.

Findings

DART effectively discriminates multiple black-box LLMs.

DART does not depend on probabilistic features.

DART performs well in multi-candidate scenarios.

Abstract

As large language models (LLMs) generate more human-like texts, concerns about the side effects of AI-generated texts (AIGT) have grown. So, researchers have developed methods for detecting AIGT. However, two challenges remain. First, the performance of detecting black-box LLMs is low because existing models focus on probabilistic features. Second, most AIGT detectors have been tested on a single-candidate setting, which assumes that we know the origin of an AIGT and which may deviate from the real-world scenario. To resolve these challenges, we propose DART, which consists of four steps: rephrasing, semantic parsing, scoring, and multiclass classification. We conducted three experiments to test the performance of DART. The experimental result shows that DART can discriminate multiple black-box LLMs without probabilistic features and the origin of AIGT.