Exploring Adversarial Robustness of Vision Transformers in the Spectral Perspective
Gihyun Kim, Juyeop Kim, Jong-Seok Lee

TL;DR
This paper investigates the adversarial robustness of Vision Transformers from a spectral perspective, revealing their reliance on phase and low frequency information and their vulnerability to frequency-selective attacks, thus providing new insights into their robustness.
Contribution
It introduces a spectral domain attack framework for Transformers, comparing their robustness to CNNs and highlighting their spectral vulnerabilities.
Findings
Transformers rely more on phase and low frequency information.
Transformers are more vulnerable to frequency-selective attacks than CNNs.
Spectral attacks reveal new robustness insights for Transformers.
Abstract
The Vision Transformer has emerged as a powerful tool for image classification tasks, surpassing the performance of convolutional neural networks (CNNs). Recently, many researchers have attempted to understand the robustness of Transformers against adversarial attacks. However, previous researches have focused solely on perturbations in the spatial domain. This paper proposes an additional perspective that explores the adversarial robustness of Transformers against frequency-selective perturbations in the spectral domain. To facilitate comparison between these two domains, an attack framework is formulated as a flexible tool for implementing attacks on images in the spatial and spectral domains. The experiments reveal that Transformers rely more on phase and low frequency information, which can render them more vulnerable to frequency-selective attacks than CNNs. This work offers new…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Exploring Adversarial Robustness of Vision Transformers in the Spectral Perspective· youtube
Taxonomy
TopicsAdversarial Robustness in Machine Learning · Bacillus and Francisella bacterial research · Anomaly Detection Techniques and Applications
