ChatGpt Content detection: A new approach using xlm-roberta alignment

TL;DR

This paper presents a new method for detecting AI-generated text using XLM-RoBERTa, incorporating feature analysis and achieving high accuracy across different text genres, aiding in AI ethics and transparency.

Contribution

Introduces a comprehensive detection approach with fine-tuned XLM-RoBERTa and feature analysis, advancing AI-generated text identification techniques.

Findings

High accuracy in detecting AI-generated text

Perplexity and attention features are key indicators

Robust performance across various text genres

Abstract

The challenge of separating AI-generated text from human-authored content is becoming more urgent as generative AI technologies like ChatGPT become more widely available. In this work, we address this issue by looking at both the detection of content that has been entirely generated by AI and the identification of human text that has been reworded by AI. In our work, a comprehensive methodology to detect AI- generated text using XLM-RoBERTa, a state-of-the-art multilingual transformer model. Our approach includes rigorous preprocessing, and feature extraction involving perplexity, semantic, and readability features. We fine-tuned the XLM-RoBERTa model on a balanced dataset of human and AI-generated texts and evaluated its performance. The model demonstrated high accuracy and robust performance across various text genres. Additionally, we conducted feature analysis to understand the model's decision-making process, revealing that perplexity and attention-based features are critical in differentiating between human and AI-generated texts. Our findings offer a valuable tool for maintaining academic integrity and contribute to the broader field of AI ethics by promoting transparency and accountability in AI systems. Future research directions include exploring other advanced models and expanding the dataset to enhance the model's generalizability.