Trans-defense: Transformer-based Denoiser for Adversarial Defense with Spatial-Frequency Domain Representation

TL;DR

This paper introduces a transformer-based denoising method that combines spatial and frequency domain analysis to improve neural network robustness against adversarial attacks on images.

Contribution

It proposes a novel denoising strategy integrating spatial features with wavelet-based frequency analysis using transformers, enhancing adversarial defense.

Findings

Significant accuracy improvements on MNIST, CIFAR-10, and Fashion-MNIST datasets.

Effective reduction of high-frequency corruption in attacked images.

Enhanced robustness of classifiers through combined denoising and retraining.

Abstract

In recent times, deep neural networks (DNNs) have been successfully adopted for various applications. Despite their notable achievements, it has become evident that DNNs are vulnerable to sophisticated adversarial attacks, restricting their applications in security-critical systems. In this paper, we present two-phase training methods to tackle the attack: first, training the denoising network, and second, the deep classifier model. We propose a novel denoising strategy that integrates both spatial and frequency domain approaches to defend against adversarial attacks on images. Our analysis reveals that high-frequency components of attacked images are more severely corrupted compared to their lower-frequency counterparts. To address this, we leverage Discrete Wavelet Transform (DWT) for frequency analysis and develop a denoising network that combines spatial image features with wavelets through a transformer layer. Next, we retrain the classifier using the denoised images, which enhances the classifier's robustness against adversarial attacks. Experimental results across the MNIST, CIFAR-10, and Fashion-MNIST datasets reveal that the proposed method remarkably elevates classification accuracy, substantially exceeding the performance by utilizing a denoising network and adversarial training approaches. The code is available at https://github.com/Mayank94/Trans-Defense.