Towards Universal and Black-Box Query-Response Only Attack on LLMs with QROA
Hussein Jawad, Yassine Chenik, Nicolas J.-B. Brunel

TL;DR
QROA introduces a black-box, query-only attack method on LLMs that efficiently finds adversarial suffixes to bypass safety measures, revealing significant security vulnerabilities.
Contribution
This paper presents QROA, a novel black-box attack framework that does not require internal model access or human-crafted templates, advancing adversarial attack techniques on LLMs.
Findings
Achieves over 80% attack success rate across multiple models.
Operates solely through query-response interactions without internal model info.
Identifies universal adversarial suffixes for broad applicability.
Abstract
The rapid adoption of Large Language Models (LLMs) has exposed critical security and ethical vulnerabilities, particularly their susceptibility to adversarial manipulations. This paper introduces QROA, a novel black-box jailbreak method designed to identify adversarial suffixes that can bypass LLM alignment safeguards when appended to a malicious instruction. Unlike existing suffix-based jailbreak approaches, QROA does not require access to the model's logit or any other internal information. It also eliminates reliance on human-crafted templates, operating solely through the standard query-response interface of LLMs. By framing the attack as an optimization bandit problem, QROA employs a surrogate model and token level optimization to efficiently explore suffix variations. Furthermore, we propose QROA-UNV, an extension that identifies universal adversarial suffixes for individual…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsCryptography and Data Security · Privacy-Preserving Technologies in Data · Access Control and Trust
MethodsQ-Learning
