Jailbreaking Large Language Models Through Alignment Vulnerabilities in Out-of-Distribution Settings

TL;DR

This paper introduces ObscurePrompt, a novel method for jailbreaking aligned LLMs by exploiting their fragile decision boundaries in out-of-distribution scenarios, demonstrating improved attack robustness over prior techniques.

Contribution

The paper presents a simple, effective approach to jailbreaking LLMs using obscure prompts that exploit vulnerabilities in out-of-distribution settings, advancing attack strategies.

Findings

ObscurePrompt significantly outperforms previous methods in attack success rate.

The approach remains effective against common defense mechanisms.

It reveals vulnerabilities in LLM alignment under OOD conditions.

Abstract

Recently, Large Language Models (LLMs) have garnered significant attention for their exceptional natural language processing capabilities. However, concerns about their trustworthiness remain unresolved, particularly in addressing ``jailbreaking'' attacks on aligned LLMs. Previous research predominantly relies on scenarios involving white-box LLMs or specific, fixed prompt templates, which are often impractical and lack broad applicability. In this paper, we introduce a straightforward and novel method called ObscurePrompt for jailbreaking LLMs, inspired by the observed fragile alignments in Out-of-Distribution (OOD) data. Specifically, we first formulate the decision boundary in the jailbreaking process and then explore how obscure text affects LLM's ethical decision boundary. ObscurePrompt starts with constructing a base prompt that integrates well-known jailbreaking techniques. Powerful LLMs are then utilized to obscure the original prompt through iterative transformations, aiming to bolster the attack's robustness. Comprehensive experiments show that our approach substantially improves upon previous methods in terms of attack effectiveness, maintaining efficacy against two prevalent defense mechanisms.