Detecting AI-Generated Text: Factors Influencing Detectability with Current Methods

TL;DR

This paper surveys current methods for detecting AI-generated text, analyzing factors affecting their effectiveness, and providing insights and recommendations for improving detection in various scenarios.

Contribution

It offers a comprehensive overview of state-of-the-art detection techniques, datasets, and factors influencing detectability, guiding future research and practical applications.

Findings

Watermarking and stylistic analysis impact detectability

Detection effectiveness varies across different scenarios

Practical recommendations for improving AI-generated text detection

Abstract

Large language models (LLMs) have advanced to a point that even humans have difficulty discerning whether a text was generated by another human, or by a computer. However, knowing whether a text was produced by human or artificial intelligence (AI) is important to determining its trustworthiness, and has applications in many domains including detecting fraud and academic dishonesty, as well as combating the spread of misinformation and political propaganda. The task of AI-generated text (AIGT) detection is therefore both very challenging, and highly critical. In this survey, we summarize state-of-the art approaches to AIGT detection, including watermarking, statistical and stylistic analysis, and machine learning classification. We also provide information about existing datasets for this task. Synthesizing the research findings, we aim to provide insight into the salient factors that combine to determine how "detectable" AIGT text is under different scenarios, and to make practical recommendations for future work towards this significant technical and societal challenge.