REALISTA: Realistic Latent Adversarial Attacks that Elicit LLM Hallucinations

TL;DR

REALISTA introduces a novel latent-space attack framework that generates realistic, semantically coherent prompts to effectively elicit hallucinations in large language models, surpassing previous methods.

Contribution

It proposes a new method combining discrete and continuous prompt attacks via a semantic dictionary, improving realism and attack success on large language models.

Findings

REALISTA outperforms existing realistic attack methods.

It successfully attacks large reasoning models in free-form response settings.

The framework achieves comparable or superior results on open-source LLMs.

Abstract

Large language models (LLMs) achieve strong performance across many tasks but remain vulnerable to hallucinations, motivating the need for realistic adversarial prompts that elicit such failures. We formulate hallucination elicitation as a constrained optimization problem, where the goal is to find semantically coherent adversarial prompts that are equivalent to benign user prompts. Existing methods remain limited: discrete prompt-based attacks preserve semantic equivalence and coherence but search only over a limited set of prompt variations, while continuous latent-space attacks explore a richer space but often decode into prompts that are no longer valid rephrasings. To address these limitations, we propose REALISTA, a realistic latent-space attack framework. REALISTA constructs an input-dependent dictionary of valid editing directions, each corresponding to a semantically equivalent and coherent rephrasing, and optimizes continuous combinations of these directions in latent space. This design combines the optimization flexibility of continuous attacks with the semantic realism of discrete rephrasing-based attacks. Experiments demonstrate that REALISTA achieves superior or comparable performance to state-of-the-art realistic attacks on open-source LLMs and, crucially, succeeds in attacking large reasoning models under free-form response settings, where prior realistic attacks fail. Code is available at https://github.com/Buyun-Liang/REALISTA.