Boosting Generative Adversarial Transferability with Self-supervised Vision Transformer Features

TL;DR

This paper introduces dSVA, a novel attack method leveraging self-supervised Vision Transformer features from contrastive learning and masked image modeling to significantly improve the transferability of adversarial examples across different models.

Contribution

It proposes a dual self-supervised ViT feature-based generative attack framework that enhances black-box adversarial transferability by exploiting both global and local features.

Findings

dSVA outperforms state-of-the-art methods in transferability

Exploiting both CL and MIM features boosts attack success rates

Self-supervised ViT features improve adversarial generalizability

Abstract

The ability of deep neural networks (DNNs) come from extracting and interpreting features from the data provided. By exploiting intermediate features in DNNs instead of relying on hard labels, we craft adversarial perturbation that generalize more effectively, boosting black-box transferability. These features ubiquitously come from supervised learning in previous work. Inspired by the exceptional synergy between self-supervised learning and the Transformer architecture, this paper explores whether exploiting self-supervised Vision Transformer (ViT) representations can improve adversarial transferability. We present dSVA -- a generative dual self-supervised ViT features attack, that exploits both global structural features from contrastive learning (CL) and local textural features from masked image modeling (MIM), the self-supervised learning paradigm duo for ViTs. We design a novel generative training framework that incorporates a generator to create black-box adversarial examples, and strategies to train the generator by exploiting joint features and the attention mechanism of self-supervised ViTs. Our findings show that CL and MIM enable ViTs to attend to distinct feature tendencies, which, when exploited in tandem, boast great adversarial generalizability. By disrupting dual deep features distilled by self-supervised ViTs, we are rewarded with remarkable black-box transferability to models of various architectures that outperform state-of-the-arts. Code available at https://github.com/spencerwooo/dSVA.