A Cross-Modal Prompt Injection Attack against Large Vision-Language Models with Image-Only Perturbation

TL;DR

This paper introduces CrossMPI, a novel attack that uses image-only perturbations to influence both text and image interpretations in large vision-language models, revealing new vulnerabilities.

Contribution

The authors propose a cross-modal prompt injection attack leveraging model hidden states and novel optimization strategies, advancing the understanding of multimodal attack surfaces.

Findings

CrossMPI outperforms baseline methods across multiple LVLMs and datasets.

Optimal attack layers are located in the middle of the model, not the last.

The attack effectively steers model interpretation of both text and images.

Abstract

Large vision-language models (LVLMs) have emerged as a powerful paradigm for multimodal intelligence, but their growing deployment also expands the attack surface of prompt injection. Despite this growing concern, existing attacks still suffer from a critical limitation: the injected prompt for one modality only steers the model's interpretation of that singular input. Alternatively, these attacks remain multimodal but fail to achieve cross-modal prompt perturbation. To bridge this gap, we introduce a novel cross-modal prompt injection attack CrossMPI, which can steer the model's interpretation of both textual and visual inputs via image-only prompt injection. Our design is underpinned by the following key breakthroughs. First, we turn the focus of the injected prompt perturbation optimization from the visual embedding space (typically with only $10^5$ parameters) to the model hidden state space (for multimodal information integration and with $10^7$ parameters). Then, two strategies are adopted to mitigate the optimization challenges posed by the larger parameter space. To constrain the optimized model parameter space, we introduce a layer selection strategy that identifies the layers most critical to multimodal integration. Interestingly, deviating from the past experience, our analysis reveals that the optimal layers for LVLM prompt perturbation reside in the middle of the model rather than the last. To constrain the image perturbation space, we propose a new distance-decremental perturbation budget assignment strategy that allocates budgets decrementally as the pixel distance to semantic-critical regions increases. Extensive experiments across multiple LVLMs and datasets show that our method significantly outperforms baseline approaches.