Defend Deep Neural Networks Against Adversarial Examples via Fixed and Dynamic Quantized Activation Functions

TL;DR

This paper introduces a novel method that uses adaptive quantization of activation functions to enhance the robustness of deep neural networks against various adversarial attacks, while also reducing model complexity.

Contribution

It is the first to propose quantizing activation functions for adversarial defense and demonstrates its effectiveness through extensive experiments.

Findings

DQA significantly improves robustness against white-box attacks.

DQA enhances resistance to black-box attacks like Zeroth Order Optimization.

The approach also reduces model size and computational complexity.

Abstract

Recent studies have shown that deep neural networks (DNNs) are vulnerable to adversarial attacks. To this end, many defense approaches that attempt to improve the robustness of DNNs have been proposed. In a separate and yet related area, recent works have explored to quantize neural network weights and activation functions into low bit-width to compress model size and reduce computational complexity. In this work, we find that these two different tracks, namely the pursuit of network compactness and robustness, can be merged into one and give rise to networks of both advantages. To the best of our knowledge, this is the first work that uses quantization of activation functions to defend against adversarial examples. We also propose to train robust neural networks by using adaptive quantization techniques for the activation functions. Our proposed Dynamic Quantized Activation (DQA) is verified through a wide range of experiments with the MNIST and CIFAR-10 datasets under different white-box attack methods, including FGSM, PGD, and C & W attacks. Furthermore, Zeroth Order Optimization and substitute model-based black-box attacks are also considered in this work. The experimental results clearly show that the robustness of DNNs could be greatly improved using the proposed DQA.