Unveiling Large Language Models Generated Texts: A Multi-Level Fine-Grained Detection Framework

TL;DR

This paper introduces a multi-level, fine-grained detection framework for identifying texts generated by large language models, combining structural, semantic, and linguistic features with contrastive learning to improve accuracy and robustness.

Contribution

The paper presents a novel multi-level detection framework that integrates various linguistic features and contrastive learning to effectively identify LLM-generated texts, especially in academic contexts.

Findings

Outperforms existing detection methods with 88.56% accuracy.

Effectively detects subtle differences and paraphrased LLM texts.

Enhances robustness against evasion techniques.

Abstract

Large language models (LLMs) have transformed human writing by enhancing grammar correction, content expansion, and stylistic refinement. However, their widespread use raises concerns about authorship, originality, and ethics, even potentially threatening scholarly integrity. Existing detection methods, which mainly rely on single-feature analysis and binary classification, often fail to effectively identify LLM-generated text in academic contexts. To address these challenges, we propose a novel Multi-level Fine-grained Detection (MFD) framework that detects LLM-generated text by integrating low-level structural, high-level semantic, and deep-level linguistic features, while conducting sentence-level evaluations of lexicon, grammar, and syntax for comprehensive analysis. To improve detection of subtle differences in LLM-generated text and enhance robustness against paraphrasing, we apply two mainstream evasion techniques to rewrite the text. These variations, along with original texts, are used to train a text encoder via contrastive learning, extracting high-level semantic features of sentence to boost detection generalization. Furthermore, we leverage advanced LLM to analyze the entire text and extract deep-level linguistic features, enhancing the model's ability to capture complex patterns and nuances while effectively incorporating contextual information. Extensive experiments on public datasets show that the MFD model outperforms existing methods, achieving an MAE of 0.1346 and an accuracy of 88.56%. Our research provides institutions and publishers with an effective mechanism to detect LLM-generated text, mitigating risks of compromised authorship. Educators and editors can use the model's predictions to refine verification and plagiarism prevention protocols, ensuring adherence to standards.