Automatic detection of Gen-AI texts: A comparative framework of neural models

TL;DR

This paper compares neural network models for detecting AI-generated texts, showing supervised models outperform commercial detectors across languages and domains.

Contribution

It introduces and evaluates four neural architectures for AI text detection, providing a comprehensive comparative analysis against existing online tools.

Findings

Supervised neural models outperform commercial detectors in accuracy.

Models demonstrate robustness across multiple languages and domains.

Detection strategies have specific strengths and limitations.

Abstract

The rapid proliferation of Large Language Models has significantly increased the difficulty of distinguishing between human-written and AI generated texts, raising critical issues across academic, editorial, and social domains. This paper investigates the problem of AI generated text detection through the design, implementation, and comparative evaluation of multiple machine learning based detectors. Four neural architectures are developed and analyzed: a Multilayer Perceptron, a one-dimensional Convolutional Neural Network, a MobileNet-based CNN, and a Transformer model. The proposed models are benchmarked against widely used online detectors, including ZeroGPT, GPTZero, QuillBot, Originality.AI, Sapling, IsGen, Rephrase, and Writer. Experiments are conducted on the COLING Multilingual Dataset, considering both English and Italian configurations, as well as on an original thematic dataset focused on Art and Mental Health. Results show that supervised detectors achieve more stable and robust performance than commercial tools across different languages and domains, highlighting key strengths and limitations of current detection strategies.