Adversarial Attacks Against MLLMs via Progressive Resolution Processing and Adaptive Feature Alignment

TL;DR

This paper introduces PRAF-Attack, a novel targeted transfer-based attack framework for Multimodal Large Language Models that enhances transferability by utilizing multi-scale features and progressive resolution refinement.

Contribution

It proposes an adaptive feature alignment and progressive resolution processing strategy to improve attack transferability against black-box MLLMs, surpassing existing methods.

Findings

PRAF-Attack outperforms seven state-of-the-art baselines in transferability.

The method effectively leverages intermediate-layer features for better robustness.

Progressive resolution processing enhances attack success across multiple scales.

Abstract

Adversarial perturbations can mislead Multimodal Large Language Models (MLLMs) recognize a benign image as a specific target object, posing serious risks in safety-critical scenarios such as autonomous driving and medical diagnosis. This makes transfer-based targeted attacks crucial for understanding and improving black-box MLLM robustness. Existing transfer-based targeted attack methods typically rely on the final global features of the surrogate encoder and anchor optimization to original-resolution target crops, leading to their limited transferability and robustness. To address these challenges, we propose Progressive Resolution Processing and Adaptive Feature Alignment (PRAF-Attack), a targeted transfer-based attack framework that integrates multi-scale global semantic guidance with robust intermediate-layer local alignment. Unlike prior methods that align only the surrogate encoder's final layer, we design an adaptive feature alignment strategy that leverages intermediate representations to enhance transferability. Specifically, we introduce an adaptive intermediate layer selection mechanism to identify transferable hierarchical features across surrogate ensembles via gradient consistency, along with an adaptive patch-level optimization strategy that preserves highly correlated local regions through efficient patch filtering. To overcome the reliance on fixed original-resolution target crops, we propose a progressive resolution processing strategy that gradually refines optimization from coarse to fine, enabling the attack to better exploit target information at multiple scales and achieve stronger transferability. We evaluate PRAF-Attack on a diverse suite of black-box MLLMs, including six open-source models and six closed-source commercial APIs. Compared with seven state-of-the-art targeted attack baselines, the proposed PRAF-Attack consistently achieves superior transferability.