Language Models Optimized to Fool Detectors Still Have a Distinct Style (And How to Change It)

TL;DR

This paper investigates the robustness of stylistic features in detecting machine-generated text, showing that models optimized to evade detectors still exhibit detectable stylistic traits, especially with multiple samples.

Contribution

It introduces a stylistic feature space that remains robust against adversarial optimization and proposes a paraphrasing attack to test detector resilience.

Findings

Stylistic features can reliably detect optimized machine text.

Detection remains effective with multiple samples, despite adversarial attacks.

Single-sample detection is vulnerable to paraphrasing attacks.

Abstract

Despite considerable progress in the development of machine-text detectors, it has been suggested that the problem is inherently hard, and therefore, that stakeholders should proceed under the assumption that machine-generated text cannot be reliably detected as such. We examine a recent such claim by Nicks et al. (2024) regarding the ease with which language models can be optimized to degrade the performance of machine-text detectors, including detectors not specifically optimized against. We identify a feature space -- the stylistic feature space -- that is robust to such optimization, and show that it may be used to reliably detect samples from language models optimized to prevent detection. Furthermore, we show that even when models are explicitly optimized against stylistic detectors, detection performance remains surprisingly unaffected. We then seek to understand if stylistic detectors are inherently more robust. To study this question, we explore a new paraphrasing approach that simultaneously aims to close the gap between human writing and machine writing in stylistic feature space while avoiding detection using traditional features. We show that when only a single sample is available for detection, this attack is universally effective across all detectors considered, including those that use writing style. However, as the number of samples available for detection grows, the human and machine distributions become distinguishable. Overall, our findings underscore previous recommendations to avoid reliance on machine-text detection.