Align is not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Models

TL;DR

This paper introduces a novel multimodal universal jailbreak attack framework that exploits image-text interactions to generate undesirable outputs in MLLMs, exposing significant safety vulnerabilities.

Contribution

It presents the first unified multimodal attack method leveraging iterative interactions and transfer strategies to challenge safety in MLLMs.

Findings

Multimodal interactions are critical vulnerabilities.

Current safety measures are insufficient against sophisticated attacks.

The attack works across multiple MLLMs, causing undesirable generations.

Abstract

Large Language Models (LLMs) have evolved into Multimodal Large Language Models (MLLMs), significantly enhancing their capabilities by integrating visual information and other types, thus aligning more closely with the nature of human intelligence, which processes a variety of data forms beyond just text. Despite advancements, the undesirable generation of these models remains a critical concern, particularly due to vulnerabilities exposed by text-based jailbreak attacks, which have represented a significant threat by challenging existing safety protocols. Motivated by the unique security risks posed by the integration of new and old modalities for MLLMs, we propose a unified multimodal universal jailbreak attack framework that leverages iterative image-text interactions and transfer-based strategy to generate a universal adversarial suffix and image. Our work not only highlights the interaction of image-text modalities can be used as a critical vulnerability but also validates that multimodal universal jailbreak attacks can bring higher-quality undesirable generations across different MLLMs. We evaluate the undesirable context generation of MLLMs like LLaVA, Yi-VL, MiniGPT4, MiniGPT-v2, and InstructBLIP, and reveal significant multimodal safety alignment issues, highlighting the inadequacy of current safety mechanisms against sophisticated multimodal attacks. This study underscores the urgent need for robust safety measures in MLLMs, advocating for a comprehensive review and enhancement of security protocols to mitigate potential risks associated with multimodal capabilities.