TwinBreak: Jailbreaking LLM Security Alignments based on Twin Prompts

TL;DR

TwinBreak is a novel method that identifies and prunes safety-related parameters in large language models, effectively bypassing safety mechanisms with high success rates and minimal computational costs.

Contribution

It introduces a fine-grained analysis technique focusing on intermediate outputs to isolate safety parameters, a first in the field.

Findings

Achieves 89% to 98% success in bypassing safety mechanisms.

Works effectively across 16 LLMs from five vendors.

Requires minimal computational resources.

Abstract

Machine learning is advancing rapidly, with applications bringing notable benefits, such as improvements in translation and code generation. Models like ChatGPT, powered by Large Language Models (LLMs), are increasingly integrated into daily life. However, alongside these benefits, LLMs also introduce social risks. Malicious users can exploit LLMs by submitting harmful prompts, such as requesting instructions for illegal activities. To mitigate this, models often include a security mechanism that automatically rejects such harmful prompts. However, they can be bypassed through LLM jailbreaks. Current jailbreaks often require significant manual effort, high computational costs, or result in excessive model modifications that may degrade regular utility. We introduce TwinBreak, an innovative safety alignment removal method. Building on the idea that the safety mechanism operates like an embedded backdoor, TwinBreak identifies and prunes parameters responsible for this functionality. By focusing on the most relevant model layers, TwinBreak performs fine-grained analysis of parameters essential to model utility and safety. TwinBreak is the first method to analyze intermediate outputs from prompts with high structural and content similarity to isolate safety parameters. We present the TwinPrompt dataset containing 100 such twin prompts. Experiments confirm TwinBreak's effectiveness, achieving 89% to 98% success rates with minimal computational requirements across 16 LLMs from five vendors.