Hidden Tail: Adversarial Image Causing Stealthy Resource Consumption in Vision-Language Models

TL;DR

This paper introduces Hidden Tail, a stealthy adversarial attack on vision-language models that induces maximum-length outputs without revealing abnormal content, highlighting vulnerabilities in model efficiency defenses.

Contribution

The paper proposes a novel prompt-agnostic adversarial image attack that maximizes output length while maintaining stealthiness, advancing resource consumption attack techniques.

Findings

Outperforms existing attacks by increasing output length up to 19.2×

Achieves maximum token limit while preserving stealthiness

Demonstrates the need for improved robustness against efficiency attacks

Abstract

Vision-Language Models (VLMs) are increasingly deployed in real-world applications, but their high inference cost makes them vulnerable to resource consumption attacks. Prior attacks attempt to extend VLM output sequences by optimizing adversarial images, thereby increasing inference costs. However, these extended outputs often introduce irrelevant abnormal content, compromising attack stealthiness. This trade-off between effectiveness and stealthiness poses a major limitation for existing attacks. To address this challenge, we propose \textit{Hidden Tail}, a stealthy resource consumption attack that crafts prompt-agnostic adversarial images, inducing VLMs to generate maximum-length outputs by appending special tokens invisible to users. Our method employs a composite loss function that balances semantic preservation, repetitive special token induction, and suppression of the end-of-sequence (EOS) token, optimized via a dynamic weighting strategy. Extensive experiments show that \textit{Hidden Tail} outperforms existing attacks, increasing output length by up to 19.2$\times$ and reaching the maximum token limit, while preserving attack stealthiness. These results highlight the urgent need to improve the robustness of VLMs against efficiency-oriented adversarial threats. Our code is available at https://github.com/zhangrui4041/Hidden_Tail.