Towards Reliable Detection of LLM-Generated Texts: A Comprehensive Evaluation Framework with CUDRT

TL;DR

This paper introduces CUDRT, a comprehensive bilingual evaluation framework for detecting LLM-generated texts across diverse operations and languages, addressing limitations of existing benchmarks.

Contribution

It presents a novel, scalable evaluation framework with extensive datasets in Chinese and English, enabling in-depth analysis of detection performance across multiple LLM activities.

Findings

Framework improves detection reliability across languages.

Operational diversity impacts detection accuracy.

Multilingual training enhances cross-linguistic detection performance.

Abstract

The increasing prevalence of large language models (LLMs) has significantly advanced text generation, but the human-like quality of LLM outputs presents major challenges in reliably distinguishing between human-authored and LLM-generated texts. Existing detection benchmarks are constrained by their reliance on static datasets, scenario-specific tasks (e.g., question answering and text refinement), and a primary focus on English, overlooking the diverse linguistic and operational subtleties of LLMs. To address these gaps, we propose CUDRT, a comprehensive evaluation framework and bilingual benchmark in Chinese and English, categorizing LLM activities into five key operations: Create, Update, Delete, Rewrite, and Translate. CUDRT provides extensive datasets tailored to each operation, featuring outputs from state-of-the-art LLMs to assess the reliability of LLM-generated text detectors. This framework supports scalable, reproducible experiments and enables in-depth analysis of how operational diversity, multilingual training sets, and LLM architectures influence detection performance. Our extensive experiments demonstrate the framework's capacity to optimize detection systems, providing critical insights to enhance reliability, cross-linguistic adaptability, and detection accuracy. By advancing robust methodologies for identifying LLM-generated texts, this work contributes to the development of intelligent systems capable of meeting real-world multilingual detection challenges. Source code and dataset are available at GitHub.