IPAD: Inverse Prompt for AI Detection - A Robust and Interpretable LLM-Generated Text Detector

TL;DR

IPAD introduces a robust, interpretable framework for detecting AI-generated text by identifying potential prompts and examining text-prompt alignment, significantly improving accuracy and trustworthiness across various data scenarios.

Contribution

The paper presents IPAD, a novel inverse prompt-based framework that enhances robustness and interpretability in detecting LLM-generated texts, outperforming existing methods.

Findings

IPAD achieves 9.05% higher recall on in-distribution data.

IPAD improves AUROC by 12.93% on out-of-distribution data.

IPAD provides interpretable evidence supporting detection decisions.

Abstract

Large Language Models (LLMs) have attained human-level fluency in text generation, which complicates the distinguishing between human-written and LLM-generated texts. This increases the risk of misuse and highlights the need for reliable detectors. Yet, existing detectors exhibit poor robustness on out-of-distribution (OOD) data and attacked data, which is critical for real-world scenarios. Also, they struggle to provide interpretable evidence to support their decisions, thus undermining the reliability. In light of these challenges, we propose IPAD (Inverse Prompt for AI Detection), a novel framework consisting of a Prompt Inverter that identifies predicted prompts that could have generated the input text, and two Distinguishers that examine the probability that the input texts align with the predicted prompts. Empirical evaluations demonstrate that IPAD outperforms the strongest baselines by 9.05% (Average Recall) on in-distribution data, 12.93% (AUROC) on out-of-distribution data, and 5.48% (AUROC) on attacked data. IPAD also performs robustly on structured datasets. Furthermore, an interpretability assessment is conducted to illustrate that IPAD enhances the AI detection trustworthiness by allowing users to directly examine the decision-making evidence, which provides interpretable support for its state-of-the-art detection results.