Exploring the Limitations of Detecting Machine-Generated Text

TL;DR

This paper investigates how stylistic differences and text complexity affect the performance of machine-generated text detectors, revealing significant sensitivity and potential unreliability in current detection methods.

Contribution

It provides an empirical analysis of detection performance across various writing styles and complexities, highlighting the need to consider stylistic factors in future detection systems.

Findings

Classifiers are highly sensitive to stylistic changes.

Detection performance degrades to random levels with certain styles.

Easy-to-read texts are more likely to be misclassified.

Abstract

Recent improvements in the quality of the generations by large language models have spurred research into identifying machine-generated text. Such work often presents high-performing detectors. However, humans and machines can produce text in different styles and domains, yet the performance impact of such on machine generated text detection systems remains unclear. In this paper, we audit the classification performance for detecting machine-generated text by evaluating on texts with varying writing styles. We find that classifiers are highly sensitive to stylistic changes and differences in text complexity, and in some cases degrade entirely to random classifiers. We further find that detection systems are particularly susceptible to misclassify easy-to-read texts while they have high performance for complex texts, leading to concerns about the reliability of detection systems. We recommend that future work attends to stylistic factors and reading difficulty levels of human-written and machine-generated text.