Universal Adversarial Attacks against Closed-Source MLLMs via Target-View Routed Meta Optimization

TL;DR

This paper introduces MCRMO-Attack, a novel method for universal targeted adversarial attacks on closed-source multimodal large language models, improving attack success rates across unknown models.

Contribution

It proposes a new meta-optimization framework with multi-crop aggregation, token routing, and cross-target prior learning to enhance universal attack effectiveness.

Findings

Boosts attack success rate by +23.7% on GPT-4o

Achieves +19.9% improvement on Gemini-2.0

Outperforms existing universal attack baselines

Abstract

Targeted adversarial attacks on closed-source multimodal large language models (MLLMs) have been increasingly explored under black-box transfer, yet prior methods are predominantly sample-specific and offer limited reusability across inputs. We instead study a more stringent setting, Universal Targeted Transferable Adversarial Attacks (UTTAA), where a single perturbation must consistently steer arbitrary inputs toward a specified target across unknown commercial MLLMs. Naively adapting existing sample-wise attacks to this universal setting faces three core difficulties: (i) target supervision becomes high-variance due to target-crop randomness, (ii) token-wise matching is unreliable because universality suppresses image-specific cues that would otherwise anchor alignment, and (iii) few-source per-target adaptation is highly initialization-sensitive, which can degrade the attainable performance. In this work, we propose MCRMO-Attack, which stabilizes supervision via Multi-Crop Aggregation with an Attention-Guided Crop, improves token-level reliability through alignability-gated Token Routing, and meta-learns a cross-target perturbation prior that yields stronger per-target solutions. Across commercial MLLMs, we boost unseen-image attack success rate by +23.7\% on GPT-4o and +19.9\% on Gemini-2.0 over the strongest universal baseline.