BreakingBED -- Breaking Binary and Efficient Deep Neural Networks by Adversarial Attacks

TL;DR

This paper evaluates the robustness of various CNN compression techniques, including binarization and pruning, against multiple adversarial attacks, revealing that binary neural networks show notable resilience.

Contribution

It provides a comprehensive analysis of the robustness of uncompressed, distilled, pruned, and binarized CNNs against diverse adversarial attacks, offering new insights for defensive strategies.

Findings

Uncompressed and pruned CNNs are highly susceptible to all attack types.

Distilled CNNs are robust against white-box attacks except C&W.

Binary neural networks demonstrate increased resilience compared to other compressed models.

Abstract

Deploying convolutional neural networks (CNNs) for embedded applications presents many challenges in balancing resource-efficiency and task-related accuracy. These two aspects have been well-researched in the field of CNN compression. In real-world applications, a third important aspect comes into play, namely the robustness of the CNN. In this paper, we thoroughly study the robustness of uncompressed, distilled, pruned and binarized neural networks against white-box and black-box adversarial attacks (FGSM, PGD, C&W, DeepFool, LocalSearch and GenAttack). These new insights facilitate defensive training schemes or reactive filtering methods, where the attack is detected and the input is discarded and/or cleaned. Experimental results are shown for distilled CNNs, agent-based state-of-the-art pruned models, and binarized neural networks (BNNs) such as XNOR-Net and ABC-Net, trained on CIFAR-10 and ImageNet datasets. We present evaluation methods to simplify the comparison between CNNs under different attack schemes using loss/accuracy levels, stress-strain graphs, box-plots and class activation mapping (CAM). Our analysis reveals susceptible behavior of uncompressed and pruned CNNs against all kinds of attacks. The distilled models exhibit their strength against all white box attacks with an exception of C&W. Furthermore, binary neural networks exhibit resilient behavior compared to their baselines and other compressed variants.