Jailbreaking Large Language Models through Iterative Tool-Disguised Attacks via Reinforcement Learning

TL;DR

This paper introduces iMIST, an adaptive multi-step attack method that disguises malicious prompts as normal tool use and escalates harm through interactive dialogues, exposing vulnerabilities in current LLM safety defenses.

Contribution

The paper presents iMIST, a novel attack technique combining tool-disguised prompts and progressive escalation, revealing weaknesses in existing LLM safety measures.

Findings

iMIST achieves higher attack success rates.

It maintains low rejection rates during attacks.

Reveals critical vulnerabilities in current defenses.

Abstract

Large language models (LLMs) have demonstrated remarkable capabilities across diverse applications, however, they remain critically vulnerable to jailbreak attacks that elicit harmful responses violating human values and safety guidelines. Despite extensive research on defense mechanisms, existing safeguards prove insufficient against sophisticated adversarial strategies. In this work, we propose iMIST (\underline{i}nteractive \underline{M}ulti-step \underline{P}rogre\underline{s}sive \underline{T}ool-disguised Jailbreak Attack), a novel adaptive jailbreak method that synergistically exploits vulnerabilities in current defense mechanisms. iMIST disguises malicious queries as normal tool invocations to bypass content filters, while simultaneously introducing an interactive progressive optimization algorithm that dynamically escalates response harmfulness through multi-turn dialogues guided by real-time harmfulness assessment. Our experiments on widely-used models demonstrate that iMIST achieves higher attack effectiveness, while maintaining low rejection rates. These results reveal critical vulnerabilities in current LLM safety mechanisms and underscore the urgent need for more robust defense strategies.