On the Effect of Pruning on Adversarial Robustness

TL;DR

Pruning convolutional networks not only reduces computational cost but also enhances adversarial robustness and generalization by acting as a regularizer, offering a simple yet effective defense against adversarial attacks.

Contribution

This paper demonstrates that pruning improves adversarial robustness and generalization without specialized training, revealing a novel defense mechanism based on network capacity reduction.

Findings

Pruning increases robustness to adversarial images.

Pruning improves generalization in convolutional networks.

Pruning achieves competitive defense results using only natural images.

Abstract

Pruning is a well-known mechanism for reducing the computational cost of deep convolutional networks. However, studies have shown the potential of pruning as a form of regularization, which reduces overfitting and improves generalization. We demonstrate that this family of strategies provides additional benefits beyond computational performance and generalization. Our analyses reveal that pruning structures (filters and/or layers) from convolutional networks increase not only generalization but also robustness to adversarial images (natural images with content modified). Such achievements are possible since pruning reduces network capacity and provides regularization, which have been proven effective tools against adversarial images. In contrast to promising defense mechanisms that require training with adversarial images and careful regularization, we show that pruning obtains competitive results considering only natural images (e.g., the standard and low-cost training). We confirm these findings on several adversarial attacks and architectures; thus suggesting the potential of pruning as a novel defense mechanism against adversarial images.