Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions

TL;DR

This paper shows that simple, multi-step, multilingual interactions can effectively jailbreak large language models, enabling harmful actions, and introduces HarmScore and Speak Easy to measure and exploit this vulnerability.

Contribution

It reveals a new vulnerability in LLM safety, demonstrating that common interactions can be exploited for harm and proposing new metrics and attack frameworks.

Findings

Increased attack success rate and HarmScore with Speak Easy framework

Simple interactions can effectively elicit harmful responses from LLMs

Vulnerability exists across open-source and proprietary models

Abstract

Despite extensive safety alignment efforts, large language models (LLMs) remain vulnerable to jailbreak attacks that elicit harmful behavior. While existing studies predominantly focus on attack methods that require technical expertise, two critical questions remain underexplored: (1) Are jailbroken responses truly useful in enabling average users to carry out harmful actions? (2) Do safety vulnerabilities exist in more common, simple human-LLM interactions? In this paper, we demonstrate that LLM responses most effectively facilitate harmful actions when they are both actionable and informative--two attributes easily elicited in multi-step, multilingual interactions. Using this insight, we propose HarmScore, a jailbreak metric that measures how effectively an LLM response enables harmful actions, and Speak Easy, a simple multi-step, multilingual attack framework. Notably, by incorporating Speak Easy into direct request and jailbreak baselines, we see an average absolute increase of 0.319 in Attack Success Rate and 0.426 in HarmScore in both open-source and proprietary LLMs across four safety benchmarks. Our work reveals a critical yet often overlooked vulnerability: Malicious users can easily exploit common interaction patterns for harmful intentions.