Breaking Barriers in Physical-World Adversarial Examples: Improving Robustness and Transferability via Robust Feature

TL;DR

This paper introduces RFCoA, a novel method for creating physical-world adversarial examples with improved transferability, robustness, and stealthiness by leveraging robust feature coverage and semantic pattern minimization, applicable to complex models.

Contribution

The paper proposes RFCoA, a new perturbation technique that enhances transferability and robustness of physical adversarial examples while maintaining stealthiness, and extends applicability to large vision-language models.

Findings

RFCoA outperforms existing methods in transferability and robustness.

The approach maintains high stealthiness in adversarial examples.

Effective on large vision-language models, demonstrating broad applicability.

Abstract

As deep neural networks (DNNs) are widely applied in the physical world, many researches are focusing on physical-world adversarial examples (PAEs), which introduce perturbations to inputs and cause the model's incorrect outputs. However, existing PAEs face two challenges: unsatisfactory attack performance (i.e., poor transferability and insufficient robustness to environment conditions), and difficulty in balancing attack effectiveness with stealthiness, where better attack effectiveness often makes PAEs more perceptible. In this paper, we explore a novel perturbation-based method to overcome the challenges. For the first challenge, we introduce a strategy Deceptive RF injection based on robust features (RFs) that are predictive, robust to perturbations, and consistent across different models. Specifically, it improves the transferability and robustness of PAEs by covering RFs of other classes onto the predictive features in clean images. For the second challenge, we introduce another strategy Adversarial Semantic Pattern Minimization, which removes most perturbations and retains only essential adversarial patterns in AEsBased on the two strategies, we design our method Robust Feature Coverage Attack (RFCoA), comprising Robust Feature Disentanglement and Adversarial Feature Fusion. In the first stage, we extract target class RFs in feature space. In the second stage, we use attention-based feature fusion to overlay these RFs onto predictive features of clean images and remove unnecessary perturbations. Experiments show our method's superior transferability, robustness, and stealthiness compared to existing state-of-the-art methods. Additionally, our method's effectiveness can extend to Large Vision-Language Models (LVLMs), indicating its potential applicability to more complex tasks.