How You Prompt Matters! Even Task-Oriented Constraints in Instructions Affect LLM-Generated Text Detection

TL;DR

This study demonstrates that task-oriented constraints in user instructions significantly affect the performance of LLM-generated text detectors, causing high variability and challenging detection accuracy in realistic scenarios.

Contribution

It reveals the impact of natural, task-specific constraints on detection performance and highlights the need to consider such instructions in developing robust detectors.

Findings

Detection performance variance increases with task constraints (up to SD of 14.4 F1-score).

Constraints generally make LLM detection more difficult.

High instruction-following ability of LLMs amplifies the effect of constraints.

Abstract

To combat the misuse of Large Language Models (LLMs), many recent studies have presented LLM-generated-text detectors with promising performance. When users instruct LLMs to generate texts, the instruction can include different constraints depending on the user's need. However, most recent studies do not cover such diverse instruction patterns when creating datasets for LLM detection. In this paper, we reveal that even task-oriented constraints -- constraints that would naturally be included in an instruction and are not related to detection-evasion -- cause existing powerful detectors to have a large variance in detection performance. We focus on student essay writing as a realistic domain and manually create task-oriented constraints based on several factors for essay quality. Our experiments show that the standard deviation (SD) of current detector performance on texts generated by an instruction with such a constraint is significantly larger (up to an SD of 14.4 F1-score) than that by generating texts multiple times or paraphrasing the instruction. We also observe an overall trend where the constraints can make LLM detection more challenging than without them. Finally, our analysis indicates that the high instruction-following ability of LLMs fosters the large impact of such constraints on detection performance.