Dropping Pixels for Adversarial Robustness

TL;DR

This paper introduces a method of improving neural network robustness against adversarial attacks by training and testing with randomly subsampled images, which enhances robustness across multiple perturbation types with minimal accuracy loss.

Contribution

The paper proposes a novel pixel subsampling technique that enhances adversarial robustness without requiring adversarial training, providing a simple yet effective defense method.

Findings

Significant robustness improvement against L0, L2, and L_inf attacks.

Minimal reduction in standard accuracy.

Subsampling acts as a set of robust features.

Abstract

Deep neural networks are vulnerable against adversarial examples. In this paper, we propose to train and test the networks with randomly subsampled images with high drop rates. We show that this approach significantly improves robustness against adversarial examples in all cases of bounded L0, L2 and L_inf perturbations, while reducing the standard accuracy by a small value. We argue that subsampling pixels can be thought to provide a set of robust features for the input image and, thus, improves robustness without performing adversarial training.

Figures

Tables

Table 1. Table 1 : Results on CIFAR10 dataset. In experiment 1, model is trained and tested with original images. In experiment 2, models are trained and tested with subsampled images with drop rate of 90 % percent 90 90\% . In experiment 3, model is trained with subsampled images with drop rates chosen uniformly in [ 0 , 1 ] 0 1 [0,1] and tested on subsampled images with drop rate of 90 % percent 90 90\% .

Model	Accuracy
Experiment 1 (ResNet-110)	$93.9\%$
Experiment 2 (ResNet-20)	$75.2\%$
Experiment 2 (ResNet-56)	$79.4\%$
Experiment 2 (ResNet-110)	$81.1\%$
Experiment 3 (ResNet-110)	$83.0\%$

Equations

$$\min$$

$$f(X^{\prime })\ne y\text{mbox}orf(X^{\prime })=y_t,$$

$$X^{j+1}={\Pi }_{X+\mathcal{S}}(X^j+V^j),\text{mbox}s.t.\Vert V^j{\Vert }_p\le {ϵ}_p^{\prime },$$

$$\Delta ={\mathbb{E}}_M[{\nabla }_X\ell (X\circ M,y)\circ M].$$

Full text

1 Introduction

Deep neural networks are known to be vulnerable against adversarial examples, inputs that are intentionally designed to cause the model to make a mistake [1]. One particular type of adversarial examples for image classifiers is slightly perturbed images that are misclassified by the model, but are recognizable to humans [2, 3]. Such adversarial images are typically generated by adding a small perturbation with bounded $L_{0}$, $L_{2}$ or $L_{\infty}$ norm to legitimate inputs [4].

Several methods have been proposed for defending against adversarial examples, but later broken using adaptive iterative attacks [5, 6]. The state-of-the-art defense against adversarial examples (with bounded $L_{\infty}$ perturbation) is adversarial training, which iteratively generates adversarial examples and trains the model to classify them correctly [7, 8]. This approach, however, significantly slows down the training process and does not properly scale to large datasets [9].

Adversarial training is shown to improve robustness at the cost of reducing accuracy [8]. In [10], Tsipras et al., argued that the trade-off between adversarial robustness and standard generalization is a fundamental property of machine learning classifiers. They analyzed a binary classification problem and showed that the reduction in standard accuracy is due to the tendency of adversarially trained models to assign non-zero weights only to a small number of strongly-correlated or “robust” features. That is, such networks discard the weakly-correlated (non-robust) features that could potentially lead to better standard generalization.

In this paper, we investigate how this insight could be used to train robust classifiers without performing adversarial training. In natural image classifiers, it is not possible to identify a fixed set of robust features in pixel domain due to the position invariance of objects. As a result, the set of robust features would be different for each image. To adapt the idea of selecting robust features to natural images, we use a slightly different notion of robust features as features that are strongly-correlated with output given all other robust features. In other words, instead of selecting features that are each highly correlated with output, we select the set of features that has the highest correlation.

Image data contain high redundancy due to the strong correlation between neighboring pixels, i.e., it is possible to restore images even when a large fraction of pixels is removed [11, 12]. Therefore, conditioned on that a pixel is selected, its surrounding pixels are weakly-correlated with output, because they significantly overlap in content with the center pixel and removing them will not cause much reduction in accuracy. Hence, one straightforward approach to construct robust features is by downsampling image pixels. Since farther pixels have smaller correlation, they non-trivially contribute to model’s prediction and, thus, are considered to be robust features.

We propose to perform random (nonuniform) sampling in order to improve both accuracy and robustness. Random subsampling of pixels improves standard generalization because the model will be trained with different subsets of pixels of each image. Also, at inference time, the accuracy can be improved by averaging the prediction over multiple sampling patterns. Moreover, since the randomness is not known to the adversary, it further mitigates the attack success rate. Randomly dropping pixels is suited to defend against adversarial examples with bounded $L_{0}$ perturbation, since the model learns to recognize objects from images with missing pixels. Nevertheless, we show that it provides robustness against adversarial examples with bounded $L_{2}$ and $L_{\infty}$ perturbation as well.

In this paper, we present our preliminary work and results on using random subsampling for adversarial robustness. Our contributions are summarized in the following.

• •

We show image classifiers can be trained with inputs with reduced redundancy, through random subsampling of pixels, without significant reduction in accuracy. We show that the best results are obtained when the model is trained with subsampled images with drop rates chosen randomly in $[0,1]$.

• •

We apply the interpretability methods on models trained with subsampled images and argue that such approaches cannot explain how the model recognizes images from few pixels. We also visualize convolutional filters of the first layer of the network and show that, in this respect, the model behaves similar to a network trained using adversarial training.

• •

We evaluate adversarial robustness of the models trained with random subsamled images. Experiments are performed on GTSRB and CIFAR10 datasets and with projected gradient descent (PGD) attack [13]. We show that training with subsampled images with drop rates chosen randomly in $[0,1]$ improves the robustness against adversarial examples in all cases of bounded $L_{0}$, $L_{2}$ and $L_{\infty}$ perturbation.

2 Training with Subsampling Pixels

Natural images are high-dimensional data with high redundancy due to the strong correlation between neighboring pixels. Hence, when training image classifiers, we can potentially reduce the redundancy without significantly reducing the standard accuracy. One approach for reducing redundancy is randomly dropping pixels at a high rate. In the following, we provide the results of training and testing models with images with missing pixels.

Let $X\in[-1,1]^{d\times d\times 3}$ be a color image. Let $M\in\{0,1\}^{d\times d\times 3}$ be a mask with the same size as $X$, where $M_{i,j,k}$ is a Bernoulli random variable with mean $1-r$, i.e., elements of $M$ are equal to [math] with probability $r$ and equal to $1$ otherwise. We generate the subsampled image $X^{\prime}$ as $X^{\prime}=M\circ X$, where $\circ$ denotes Hadamard (element-wise) multiplication. Figure 1 shows samples of original images of German Traffic Sign Recognition Benchmark (GTSRB) [14] and CIFAR10 dataset [15] and their corresponding subsampled images with drop rate of $90\%$.

2.1 Experimental Results

We use ResNet-20 and ResNet-110 architectures [16] for GTSRB and CIFAR10 datasets, respectively. The models are trained and tested with subsampled images. During training, the mask is chosen randomly and differently for each image and at each epoch. Figure 2 shows accuracy of models trained with images with different drop rates. As expected, dropping pixels at a higher rate results in lower accuracy. However, even at very high drop rates, the accuracy remains high. Specifically, compared to standard training, at drop rate of $90\%$ the accuracy reduces by only about $4\%$ and $13\%$ for GTSRB and CIFAR10 datasets, respectively.

We also observed that deeper networks perform better. Table 1 shows the accuracy of ResNet models with different depths on CIFAR10 images with drop rate of $90\%$. As can be seen, ResNet-110 provides about $2\%$ and $6\%$ higher accuracy compared to ResNet-56 and ResNet-20, respectively. Moreover, the model achieves best results when the drop rate of each image is chosen randomly between $0\%$ and $100\%$ at each epoch.

2.2 Interpretability Analysis

In recent years, several “post-hoc” methods have been proposed for interpreting the predictions of deep convolutional neural networks [17, 18, 19, 20]. Such methods typically identify input dimensions that the output is most sensitive to. Let $X$ be the input image, $F$ be the classifier and $E$ be the explanation function that maps inputs to objects of the same shape. Most explanation methods are based on some form of the gradient of the classifier function with respect to input [21]. In our analysis, we use magnitude of gradient as the explanation map, i.e., $E(X)=|\partial F(X)/\partial X|$ [17].

We examine the interpretability for a ResNet-110 network trained with subsampled CIFAR10 images with drop rates chosen randomly in $[0,1]$. Figure 3(a) shows explanation maps $E(X)$ and $E(X^{\prime})$ for original and subsampled images, respectively. For original images, the explanation map is similar to the pattern of edges in image, a phenomenon that [21] also observed and posed as a shortcoming of interpretability methods. For subsampled images, however, the explanation is not informative. We visualize $E(X^{\prime})\circ M$ and $E(X^{\prime})\circ(1-M)$, which respectively show the gradient magnitude at pixels that have been dropped and those that are not dropped. As can be seen, most of larger values of gradient are at positions of dropped pixels, i.e., pixels that do not contribute to the model prediction.

The results raise questions about the usability of such techniques in explaining model predictions. The gradient captures the sensitivity of the model output with respect to its input, i.e., it quantifies how much a change in a small neighborhood around the input would change the predictions $F(X)$. It, however, does not quantify how much each input dimension contributes to the model prediction. Specifically, in our case, such interpretability methods do not explain how the model recognizes the image from few pixels.

For classifying subsampled images, the network might implicitly rely on features of original images, i.e., it might have learned to produce similar representations for original and subsampled images. In order to prevent the model to do so, we train a model to classify subsampled images into their true label, while mapping original images to uniform distribution. This training approach results in a network with accuracy of $78.9\%$ on subsampled images (with $90\%$ drop rate), which is only about $2\%$ less than a model that is only trained with subsampled images. The results imply that the network is capable of classifying subsampled images without actually learning features of natural images. Figure 3(b) shows the explanation maps for few images. Similar to 3(a), the explanations do not provide insights into the model workflow.

Finally, we train a model to classify subsampled images and subsampled noisy images differently to investigate to what extent the network relies on the exact values of subsampled pixels. Specifically, we train the model to classify subsampled images into their true label, while mapping subsampled noisy images to uniform distribution. The subsampled noisy images are obtained as $X^{\prime}=(X+\epsilon V)\circ M$, where $M$ is sampling mask, $V$ is a random variable that takes values of $\{-1,1\}$ with equal probability and $\epsilon=16/255$. The drop rate is $90\%$.

Interestingly, the trained model achieves accuracy of $80.9\%$ on subsampled images, which is almost the same as a model trained only with subsampled images. Figure 3(c) shows the explanation maps for few images. For this model, explanations on original images are not correlated with edge pattern. Also, explanations on subsampled images are sparser compared to 3(a) and 3(b). Moreover, most of larger values of gradient are at positions where pixels have not been dropped. Further exploring interpretability of networks trained with subsampled images is left for future work.

2.3 Visualizing Convolutional Filters

Convolutional networks are known to learn basic image patterns such as edges and blobs in early layers and then combine them in later layers to distinguish complex objects [22]. Dropping pixels at a high rate disrupts such basic patterns. As a result, the network will not be able to readily extract spatial features of the image data. To gain insight into how the model classifies inputs, we examine convolutional filters of the first layer. Figure 4 shows the visualization of filters of ResNet-110 networks trained with CIFAR10 dataset. We consider three cases of a normally trained model, a model trained with subsampled images with drop rate of $90\%$ and a model trained with subsampled images with drop rates chosen randomly in $[0,1]$.

As can be seen, the model that is trained with subsampled images only has filters with large values at center position. This means that the network recognizes that there is no spatial correlation between adjacent pixels and, hence, just passes several scaled versions of the image to the next layer. Also, the model that is trained with subsampled images with different drop rates contains a mix of concentrated filters and other filters similar to the normally trained model.

Interestingly, having concentrated filters in first layer is also observed in adversarially trained networks on MNIST dataset, where models were trained with adversarial examples with bounded $L_{\infty}$ perturbation [8]. The similar behavior of the models trained with the two approaches suggests that randomly dropping pixels is indeed related to the notion of robust features observed in adversarial training. Further exploring this relationship is left for the future work.

3 Robustness to Adversarial Examples

In this section, we first provide a background on adversarial examples and then evaluate the robustness of models trained with subsampled images.

3.1 Background on Adversarial Examples

We consider a class of adversarial examples for image classifiers where small (imperceptible) perturbation is added to an image so that the model misclassifies it (misclassification attack) or classifies it into the attacker’s desired label (targeted attack). The perturbation is typically quantified according to an $L_{p}$ norm. The attacker’s problem is formally stated as follows:

[TABLE]

where $X$ and $X^{\prime}$ are the clean and adversarial examples, respectively, $y$ is the true label and $y_{t}\neq y$ is the attacker’s desired target label.

We generate adversarial examples using the Projected Gradient Descent (PGD) method [13, 8], such that the added perturbation is bounded within $\epsilon_{p}$ for $L_{p}$ norm, i.e., $\|X^{\prime}-X\|_{p}\leq\epsilon_{p}$. PGD is an iterative attack with the following update step:

[TABLE]

where $X^{j}$ is the image at step $j$, $V^{j}$ is the attack vector at step $j$, $\epsilon^{\prime}_{p}$ is the added perturbation per step, and $\Pi_{X+\mathcal{S}}$ is the projection operator where $\mathcal{S}$ is the set of allowed perturbations. According to attack goal, the attack vector is specified as follows:

• •

$V^{j}=\nabla_{X}\ell(X^{j},y)$, for misclassification attack,

• •

$V^{j}=-\nabla_{X}\ell(X^{j},y_{t})$, for targeted attack.

Attack Setup. To attack a model with random subsampling, we generate $10$ randomly subsampled images, for each one compute the gradient at pixels which have not been dropped, and then take average of gradients. Let $M$ be the sampling mask. The average gradient is formally obtained as follows:

[TABLE]

We consider the cases where the $L_{0}$, $L_{2}$ or $L_{\infty}$ norm of perturbation is bounded. For GTSRB, we set $\epsilon_{0}=0.03\hskip 2.84544ptD$, where $D$ is the total number of pixels, $\epsilon_{2}=512/255$ and $\epsilon_{\infty}=32/255$. The attack step size is set to $\epsilon^{\prime}_{0}=1$, $\epsilon^{\prime}_{2}=32/255$ and $\epsilon^{\prime}_{\infty}=4/255$. For CIFAR10, we set $\epsilon_{0}=0.03\hskip 2.84544ptD$, $\epsilon_{2}=1$ and $\epsilon_{\infty}=16/255$. The attack step size is also set to $\epsilon^{\prime}_{0}=1$, $\epsilon^{\prime}_{2}=16/255$ and $\epsilon^{\prime}_{\infty}=2/255$. We perform PGD attack with cross-entropy and CW [4] loss functions and for misclassification and targeted attacks, and present the best attack results.

3.2 Case of Bounded $L_{0}$ Perturbation

Training with random subsampling is suited to defend against $L_{0}$ adversarial examples, since the model is trained to be robust to missing pixels. We also observed that subsampled adversarial examples result in more distributed output probability vector than subsampled clean images. Therefore, we enhance the defense mechanism by rejecting examples for which the entropy of probability vector is larger than a threshold. The threshold is chosen so as to have $1\%$ false positive rate on validation data, i.e., the accuracy on clean validation images is reduced by $1\%$.

With random subsampling, the accuracy can be improved by averaging the output on multiple different subsampled versions of the input. To improve adversarial robustness, we compute the average output probability vector over $10$ different subsampled inputs and reject the example if the entropy of probability vector is larger than a threshold. The threshold is chosen such that accuracy on clean validation images is $1\%$ less than the case where each example is run only once.

Figure 5 shows the results on GTSRB and CIFAR10 datasets. As expected, larger drop rate improves adversarial robustness at the cost of reducing standard accuracy. The experiments are performed on a single model which has been trained with images with drop rates in $[0,1]$. Such a model has the advantage that at test time the drop rate can be tuned to achieve different levels of trade-offs between accuracy and robustness.

In Figure 5, it can be also seen that rejecting inputs based on the value of entropy improves the trade-off of accuracy and robustness, since the adversarial accuracy is increased by larger value than the $1\%$ reduction in standard accuracy. Moreover, averaging over $10$ runs and rejecting inputs with high entropy results in adversarial accuracy to be on par with standard accuracy at high drop rates.

3.3 Case of Bounded $L_{2}$ and $L_{\infty}$ Perturbation

Figure 6 shows the attack results for $L_{2}$ and $L_{\infty}$ cases for GTSRB and CIFAR10 datasets. As can be seen, random subsampling improves robustness in both cases and, similar to $L_{0}$ case, larger drop rate results in higher adversarial robustness. Intuitively, for attacking the model, the adversary needs to distribute their budget to all features in such a way that the expected attack success rate is maximized. This will reduce the attack effectiveness compared to the case that the attacker knows the exact sampling pattern.

4 Related Work

Defenses against adversarial examples with bounded $L_{p}$ perturbation have been widely studied [7, 4, 5, 8, 6, 23]. Adversarial training is the state-of-the-art approach for $L_{2}$ and $L_{\infty}$ cases, but is shown to significantly slow down the training procedure [8, 10]. While most papers studied defenses in $L_{\infty}$ setting, some of real-world attacks based on adversarial examples fit in the $L_{0}$ setting [24, 25]. For example, [24] attacked traffic sign detection algorithms by adding sticker-like perturbation to images. Also, [25] showed face recognition algorithms can be fooled by adding physically-realizable perturbation such as eyeglasses to images.

In [26], the authors proposed a method for improving robustness in $L_{0}$ attack setting by exploiting the sparsity of natural images in Fourier domain. They showed attack results on MNIST and Fashion-MNIST datasets and mentioned that the sparsity property might not hold in large images. Similar to their method, we used a property of natural images, namely high spatial correlation, to mitigate the effect of adversarial perturbation. Our approach is, however, general to natural images of any size. In fact, with larger images, it is possible to drop pixels at a higher rate and still restore the image [11, 12]. Hence, the classifier might be able to recognize subsampled images with higher drop rates and, as a result, achieve better robustness. Moreover, our method improves the robustness against $L_{2}$ and $L_{\infty}$ adversarial examples in addition to the $L_{0}$ case. As a future work, we will implement our method on Imagenet dataset.

Several papers have proposed using post-processing algorithms to increase adversarial robustness [27, 28]. In [27], the authors proposed applying random resizing and padding at inference time. [28] presented an algorithm for pruning a random subset of activations of a pretrained networks and scaling up the rest. Unlike our method, such algorithms do not train the model to learn the randomness.

Introducing randomness to inputs or the network itself at both training and test times is recently explored and shown to improve performance on adversarial examples [29, 30]. In [29], the authors proposed adding random noise layers to the network and ensembling the prediction over random noises. [30] adopted similar idea and used differential privacy to provide certified robustness against adversarial perturbations. In this paper, we proposed to train the model with subsampled images, with the drop rates randomly chosen in $[0,1]$, and test it with subsampled images with high drop rates. We showed that our method improves adversarial robustness in all cases of $L_{0}$, $L_{2}$ and $L_{\infty}$ perturbations.

5 Conclusion

In this paper, we showed that image classifiers can be trained to recognize images with high drop rates. We then proposed to train models with subsampled images with drop rates randomly chosen in $[0,1]$. Our experimental results on GTSRB and CIFR10 datasets showed that such models improve the robustness against adversarial examples in all cases of $L_{0}$, $L_{2}$ and $L_{\infty}$ perturbation, while reducing standard accuracy by a small value.

Acknowledgments

This work was supported by ONR grant N00014-17-S-B001.