Attention! Your Vision Language Model Could Be Maliciously Manipulated

TL;DR

This paper reveals the vulnerability of Vision-Language Models to subtle adversarial image attacks, introduces a new attack method called VMA, and demonstrates its effectiveness in various malicious and protective applications.

Contribution

The paper introduces VMA, a novel attack method combining optimization techniques to manipulate VLM outputs and also to embed watermarks, highlighting both security risks and potential safeguards.

Findings

VMA effectively manipulates VLM outputs with imperceptible perturbations.

VMA demonstrates versatility in attacks like jailbreaking, hijacking, and privacy breaches.

VMA can also embed watermarks for copyright protection.

Abstract

Large Vision-Language Models (VLMs) have achieved remarkable success in understanding complex real-world scenarios and supporting data-driven decision-making processes. However, VLMs exhibit significant vulnerability against adversarial examples, either text or image, which can lead to various adversarial outcomes, e.g., jailbreaking, hijacking, and hallucination, etc. In this work, we empirically and theoretically demonstrate that VLMs are particularly susceptible to image-based adversarial examples, where imperceptible perturbations can precisely manipulate each output token. To this end, we propose a novel attack called Vision-language model Manipulation Attack (VMA), which integrates first-order and second-order momentum optimization techniques with a differentiable transformation mechanism to effectively optimize the adversarial perturbation. Notably, VMA can be a double-edged sword: it can be leveraged to implement various attacks, such as jailbreaking, hijacking, privacy breaches, Denial-of-Service, and the generation of sponge examples, etc, while simultaneously enabling the injection of watermarks for copyright protection. Extensive empirical evaluations substantiate the efficacy and generalizability of VMA across diverse scenarios and datasets. Code is available at https://github.com/Trustworthy-AI-Group/VMA.