Improving Robustness of Convolutional Neural Networks Using Element-Wise Activation Scaling

TL;DR

This paper introduces Element-Wise Activation Scaling (EWAS), a novel method that individually adjusts each activation element in CNNs to significantly enhance adversarial robustness, outperforming previous channel-level approaches.

Contribution

EWAS is a new fine-grained activation manipulation technique that improves CNN robustness against adversarial attacks more effectively than prior channel-level methods.

Findings

EWAS increases adversarial accuracy by 37.65% on ResNet-18 with CIFAR10.

EWAS significantly outperforms previous channel-level activation scaling methods.

The method is simple, effective, and applicable to different CNN architectures.

Abstract

Recent works reveal that re-calibrating the intermediate activation of adversarial examples can improve the adversarial robustness of a CNN model. The state of the arts [Baiet al., 2021] and [Yanet al., 2021] explores this feature at the channel level, i.e. the activation of a channel is uniformly scaled by a factor. In this paper, we investigate the intermediate activation manipulation at a more fine-grained level. Instead of uniformly scaling the activation, we individually adjust each element within an activation and thus propose Element-Wise Activation Scaling, dubbed EWAS, to improve CNNs' adversarial robustness. Experimental results on ResNet-18 and WideResNet with CIFAR10 and SVHN show that EWAS significantly improves the robustness accuracy. Especially for ResNet18 on CIFAR10, EWAS increases the adversarial accuracy by 37.65% to 82.35% against C&W attack. EWAS is simple yet very effective in terms of improving robustness. The codes are anonymously available at https://anonymous.4open.science/r/EWAS-DD64.