GPT-4 Jailbreaks Itself with Near-Perfect Success Using Self-Explanation

TL;DR

This paper introduces IRIS, a novel self-explanation based method that effectively jailbreaks large language models like GPT-4 with high success rates and minimal queries, highlighting security vulnerabilities.

Contribution

IRIS is the first approach to use self-explanation for iterative prompt refinement in black-box jailbreaking, achieving near-perfect success rates with fewer queries.

Findings

IRIS achieves 98% success on GPT-4.

IRIS outperforms prior methods in efficiency and success rate.

IRIS requires fewer than 7 queries for effective jailbreaking.

Abstract

Research on jailbreaking has been valuable for testing and understanding the safety and security issues of large language models (LLMs). In this paper, we introduce Iterative Refinement Induced Self-Jailbreak (IRIS), a novel approach that leverages the reflective capabilities of LLMs for jailbreaking with only black-box access. Unlike previous methods, IRIS simplifies the jailbreaking process by using a single model as both the attacker and target. This method first iteratively refines adversarial prompts through self-explanation, which is crucial for ensuring that even well-aligned LLMs obey adversarial instructions. IRIS then rates and enhances the output given the refined prompt to increase its harmfulness. We find that IRIS achieves jailbreak success rates of 98% on GPT-4, 92% on GPT-4 Turbo, and 94% on Llama-3.1-70B in under 7 queries. It significantly outperforms prior approaches in automatic, black-box, and interpretable jailbreaking, while requiring substantially fewer queries, thereby establishing a new standard for interpretable jailbreaking methods.