LLM Detectors Still Fall Short of Real World: Case of LLM-Generated Short News-Like Posts

TL;DR

Existing LLM detectors are ineffective in real-world scenarios involving short news-like posts, showing vulnerability to simple attacks and poor generalization, highlighting the need for improved, domain-specific benchmarking methods.

Contribution

This paper demonstrates the limitations of current LLM detectors in real-world settings and proposes a new, extensible benchmark for evaluating their robustness and generalization.

Findings

Zero-shot detectors perform inconsistently and are vulnerable to temperature attacks.

Purpose-trained detectors struggle to generalize to new human-written texts.

Benchmarking approaches need re-evaluation to better reflect real-world challenges.

Abstract

With the emergence of widely available powerful LLMs, disinformation generated by large Language Models (LLMs) has become a major concern. Historically, LLM detectors have been touted as a solution, but their effectiveness in the real world is still to be proven. In this paper, we focus on an important setting in information operations -- short news-like posts generated by moderately sophisticated attackers. We demonstrate that existing LLM detectors, whether zero-shot or purpose-trained, are not ready for real-world use in that setting. All tested zero-shot detectors perform inconsistently with prior benchmarks and are highly vulnerable to sampling temperature increase, a trivial attack absent from recent benchmarks. A purpose-trained detector generalizing across LLMs and unseen attacks can be developed, but it fails to generalize to new human-written texts. We argue that the former indicates domain-specific benchmarking is needed, while the latter suggests a trade-off between the adversarial evasion resilience and overfitting to the reference human text, with both needing evaluation in benchmarks and currently absent. We believe this suggests a re-consideration of current LLM detector benchmarking approaches and provides a dynamically extensible benchmark to allow it (https://github.com/Reliable-Information-Lab-HEVS/benchmark_llm_texts_detection).