AI Generated Text Detection Using Instruction Fine-tuned Large Language and Transformer-Based Models

TL;DR

This paper develops detection methods for AI-generated text using instruction fine-tuned large language models and transformers, addressing challenges in distinguishing machine from human writing and identifying the source model.

Contribution

It introduces fine-tuning approaches for LLMs and transformers to improve detection accuracy of AI-generated texts and source identification.

Findings

Achieved 95.47% accuracy in distinguishing human vs. machine text.

Successfully identified the source LLM with 46.98% accuracy.

Demonstrated effectiveness of fine-tuned models in AI text detection.

Abstract

Large Language Models (LLMs) possess an extraordinary capability to produce text that is not only coherent and contextually relevant but also strikingly similar to human writing. They adapt to various styles and genres, producing content that is both grammatically correct and semantically meaningful. Recently, LLMs have been misused to create highly realistic phishing emails, spread fake news, generate code to automate cyber crime, and write fraudulent scientific articles. Additionally, in many real-world applications, the generated content including style and topic and the generator model are not known beforehand. The increasing prevalence and sophistication of artificial intelligence (AI)-generated texts have made their detection progressively more challenging. Various attempts have been made to distinguish machine-generated text from human-authored content using linguistic, statistical, machine learning, and ensemble-based approaches. This work focuses on two primary objectives Task-A, which involves distinguishing human-written text from machine-generated text, and Task-B, which attempts to identify the specific LLM model responsible for the generation. Both of these tasks are based on fine tuning of Generative Pre-trained Transformer (GPT_4o-mini), Large Language Model Meta AI (LLaMA) 3 8B, and Bidirectional Encoder Representations from Transformers (BERT). The fine-tuned version of GPT_4o-mini and the BERT model has achieved accuracies of 0.9547 for Task-A and 0.4698 for Task-B.