"I know myself better, but not really greatly": How Well Can LLMs Detect and Explain LLM-Generated Texts?

TL;DR

This study evaluates the ability of current LLMs to detect and explain LLM-generated texts, revealing significant limitations and the potential of ternary classification to improve performance and interpretability.

Contribution

The paper provides a comprehensive analysis of LLM detection and explanation capabilities, introducing a ternary classification framework and identifying key explanation failures.

Findings

Self-detection outperforms cross-detection.

Ternary classification improves detection accuracy.

Explanation quality is hindered by hallucinations and flawed reasoning.

Abstract

Distinguishing between human- and LLM-generated texts is crucial given the risks associated with misuse of LLMs. This paper investigates detection and explanation capabilities of current LLMs across two settings: binary (human vs. LLM-generated) and ternary classification (including an ``undecided'' class). We evaluate 6 close- and open-source LLMs of varying sizes and find that self-detection (LLMs identifying their own outputs) consistently outperforms cross-detection (identifying outputs from other LLMs), though both remain suboptimal. Introducing a ternary classification framework improves both detection accuracy and explanation quality across all models. Through comprehensive quantitative and qualitative analyses using our human-annotated dataset, we identify key explanation failures, primarily reliance on inaccurate features, hallucinations, and flawed reasoning. Our findings underscore the limitations of current LLMs in self-detection and self-explanation, highlighting the need for further research to address overfitting and enhance generalizability.