Re-Triggering Safeguards within LLMs for Jailbreak Detection

TL;DR

This paper introduces an embedding disruption method to re-trigger safeguards in large language models, effectively detecting and defending against jailbreak prompts in various attack scenarios.

Contribution

It presents a novel approach that cooperates with LLMs' internal defenses by re-activating safeguards through embedding disruption, improving jailbreak detection.

Findings

Effectively defends against state-of-the-art jailbreak attacks

Robust in both white-box and black-box settings

Remains effective against adaptive attacks

Abstract

This paper proposes a jailbreaking prompt detection method for large language models (LLMs) to defend against jailbreak attacks. Although recent LLMs are equipped with built-in safeguards, it remains possible to craft jailbreaking prompts that bypass them. We argue that such jailbreaking prompts are inherently fragile, and thus introduce an embedding disruption method to re-activate the safeguards within LLMs. Unlike previous defense methods that aim to serve as standalone solutions, our approach instead cooperates with the LLM's internal defense mechanisms by re-triggering them. Moreover, through extensive analysis, we gain a comprehensive understanding of the disruption effects and develop an efficient search algorithm to identify appropriate disruptions for effective jailbreak detection. Extensive experiments demonstrate that our approach effectively defends against state-of-the-art jailbreak attacks in white-box and black-box settings, and remains robust even against adaptive attacks.