Investigating the Impact of Quantization on Adversarial Robustness

TL;DR

This paper investigates how different quantization techniques and stages affect the robustness of deep models against adversarial attacks, revealing inconsistencies in prior studies and offering insights for secure deployment.

Contribution

It provides the first comprehensive analysis of quantization pipeline components' impact on adversarial robustness, clarifying conflicting previous findings.

Findings

Quantization stage and robust optimization choice influence model robustness.

Inconsistent conclusions in prior work stem from different pipeline configurations.

Guidelines for deploying more secure quantized models in resource-limited, high-security scenarios.

Abstract

Quantization is a promising technique for reducing the bit-width of deep models to improve their runtime performance and storage efficiency, and thus becomes a fundamental step for deployment. In real-world scenarios, quantized models are often faced with adversarial attacks which cause the model to make incorrect inferences by introducing slight perturbations. However, recent studies have paid less attention to the impact of quantization on the model robustness. More surprisingly, existing studies on this topic even present inconsistent conclusions, which prompted our in-depth investigation. In this paper, we conduct a first-time analysis of the impact of the quantization pipeline components that can incorporate robust optimization under the settings of Post-Training Quantization and Quantization-Aware Training. Through our detailed analysis, we discovered that this inconsistency arises from the use of different pipelines in different studies, specifically regarding whether robust optimization is performed and at which quantization stage it occurs. Our research findings contribute insights into deploying more secure and robust quantized networks, assisting practitioners in reference for scenarios with high-security requirements and limited resources.