A General Method for Detecting Information Generated by Large Language Models

TL;DR

This paper presents a novel general detection method for identifying LLM-generated content across unseen models and domains, addressing limitations of existing approaches and enhancing digital trust.

Contribution

We introduce a general LLM detector (GLD) that combines twin memory networks and a theory-guided module for improved generalization to new LLMs and domains.

Findings

GLD outperforms existing detection methods in empirical tests.

The approach effectively detects content from unseen LLMs.

Case studies confirm practical applicability.

Abstract

The proliferation of large language models (LLMs) has significantly transformed the digital information landscape, making it increasingly challenging to distinguish between human-written and LLM-generated content. Detecting LLM-generated information is essential for preserving trust on digital platforms (e.g., social media and e-commerce sites) and preventing the spread of misinformation, a topic that has garnered significant attention in IS research. However, current detection methods, which primarily focus on identifying content generated by specific LLMs in known domains, face challenges in generalizing to new (i.e., unseen) LLMs and domains. This limitation reduces their effectiveness in real-world applications, where the number of LLMs is rapidly multiplying and content spans a vast array of domains. In response, we introduce a general LLM detector (GLD) that combines a twin memory networks design and a theory-guided detection generalization module to detect LLM-generated information across unseen LLMs and domains. Using real-world datasets, we conduct extensive empirical evaluations and case studies to demonstrate the superiority of GLD over state-of-the-art detection methods. The study has important academic and practical implications for digital platforms and LLMs.