Arondight: Red Teaming Large Vision Language Models with Auto-generated Multi-modal Jailbreak Prompts

TL;DR

This paper introduces Arondight, a comprehensive red teaming framework for evaluating the security of large vision language models (VLMs), revealing significant vulnerabilities in generating harmful content and proposing methods to improve safety assessments.

Contribution

We develop Arondight, a novel automated multi-modal red team framework for VLMs, incorporating reinforcement learning guided prompt generation and diversity metrics, addressing gaps in existing security evaluation methods.

Findings

Achieved an 84.5% attack success rate on GPT-4 in toxic prompt scenarios.

Exposed significant vulnerabilities in current VLMs regarding harmful content generation.

Provided a categorized safety level assessment and reinforcement learning recommendations for VLMs.

Abstract

Large Vision Language Models (VLMs) extend and enhance the perceptual abilities of Large Language Models (LLMs). Despite offering new possibilities for LLM applications, these advancements raise significant security and ethical concerns, particularly regarding the generation of harmful content. While LLMs have undergone extensive security evaluations with the aid of red teaming frameworks, VLMs currently lack a well-developed one. To fill this gap, we introduce Arondight, a standardized red team framework tailored specifically for VLMs. Arondight is dedicated to resolving issues related to the absence of visual modality and inadequate diversity encountered when transitioning existing red teaming methodologies from LLMs to VLMs. Our framework features an automated multi-modal jailbreak attack, wherein visual jailbreak prompts are produced by a red team VLM, and textual prompts are generated by a red team LLM guided by a reinforcement learning agent. To enhance the comprehensiveness of VLM security evaluation, we integrate entropy bonuses and novelty reward metrics. These elements incentivize the RL agent to guide the red team LLM in creating a wider array of diverse and previously unseen test cases. Our evaluation of ten cutting-edge VLMs exposes significant security vulnerabilities, particularly in generating toxic images and aligning multi-modal prompts. In particular, our Arondight achieves an average attack success rate of 84.5\% on GPT-4 in all fourteen prohibited scenarios defined by OpenAI in terms of generating toxic text. For a clearer comparison, we also categorize existing VLMs based on their safety levels and provide corresponding reinforcement recommendations. Our multimodal prompt dataset and red team code will be released after ethics committee approval. CONTENT WARNING: THIS PAPER CONTAINS HARMFUL MODEL RESPONSES.