Divide and Conquer: A Hybrid Strategy Defeats Multimodal Large Language Models

TL;DR

This paper introduces JMLLM, a multimodal jailbreaking approach that effectively exploits vulnerabilities in large language models across text, visual, and auditory modalities, supported by a new dataset and extensive experiments.

Contribution

It presents a novel multimodal jailbreaking method and a comprehensive dataset, advancing security testing for large language models across multiple modalities.

Findings

High attack success rates across 13 LLMs

Significant reduction in attack time overhead

Enhanced coverage of jailbreak modalities

Abstract

Large language models (LLMs) are widely applied in various fields of society due to their powerful reasoning, understanding, and generation capabilities. However, the security issues associated with these models are becoming increasingly severe. Jailbreaking attacks, as an important method for detecting vulnerabilities in LLMs, have been explored by researchers who attempt to induce these models to generate harmful content through various attack methods. Nevertheless, existing jailbreaking methods face numerous limitations, such as excessive query counts, limited coverage of jailbreak modalities, low attack success rates, and simplistic evaluation methods. To overcome these constraints, this paper proposes a multimodal jailbreaking method: JMLLM. This method integrates multiple strategies to perform comprehensive jailbreak attacks across text, visual, and auditory modalities. Additionally, we contribute a new and comprehensive dataset for multimodal jailbreaking research: TriJail, which includes jailbreak prompts for all three modalities. Experiments on the TriJail dataset and the benchmark dataset AdvBench, conducted on 13 popular LLMs, demonstrate advanced attack success rates and significant reduction in time overhead.