Why AI-Generated Text Detection Fails: Evidence from Explainable AI Beyond Benchmark Accuracy

TL;DR

This paper critically examines the reliability of AI-generated text detectors, revealing their dependence on dataset-specific cues and limited generalization, despite high benchmark scores, and proposes an explainable, linguistics-based detection framework.

Contribution

The authors introduce an interpretable detection framework using linguistic features and explainable AI, highlighting the limitations of current models in cross-domain robustness.

Findings

Detectors perform well on in-domain data with F1 score of 0.9734.

Significant performance degradation occurs under domain shift and across different generators.

Detectors rely on dataset-specific stylistic cues rather than stable indicators of machine authorship.

Abstract

The widespread adoption of Large Language Models (LLMs) has made the detection of AI-Generated text a pressing and complex challenge. Although many detection systems report high benchmark accuracy, their reliability in real-world settings remains uncertain, and their interpretability is often unexplored. In this work, we investigate whether contemporary detectors genuinely identify machine authorship or merely exploit dataset-specific artefacts. We propose an interpretable detection framework that integrates linguistic feature engineering, machine learning, and explainable AI techniques. When evaluated on two prominent benchmark corpora, namely PAN CLEF 2025 and COLING 2025, our model trained on 30 linguistic features achieves leaderboard-competitive performance, attaining an F1 score of 0.9734. However, systematic cross-domain and cross-generator evaluation reveals substantial generalisation failure: classifiers that excel in-domain degrade significantly under distribution shift. Using SHAP- based explanations, we show that the most influential features differ markedly between datasets, indicating that detectors often rely on dataset-specific stylistic cues rather than stable signals of machine authorship. Further investigation with in-depth error analysis exposes a fundamental tension in linguistic-feature-based AI text detection: the features that are most discriminative on in-domain data are also the features most susceptible to domain shift, formatting variation, and text-length effects. We believe that this knowledge helps build AI detectors that are robust across different settings. To support replication and practical use, we release an open-source Python package that returns both predictions and instance-level explanations for individual texts.