ODG-Q: Robust Quantization via Online Domain Generalization

TL;DR

This paper introduces ODG-Q, a novel robust quantization method that enhances neural network resilience against adversarial attacks by using online domain generalization, achieving significant improvements with low training costs.

Contribution

The paper presents ODG-Q, a new approach that recasts robust quantization as an online domain generalization problem, improving adversarial robustness efficiently.

Findings

49.2% average improvement under white-box attacks on CIFAR-10

21.7% average improvement under black-box attacks on CIFAR-10

First to train robust quantized and binary networks on ImageNet

Abstract

Quantizing neural networks to low-bitwidth is important for model deployment on resource-limited edge hardware. Although a quantized network has a smaller model size and memory footprint, it is fragile to adversarial attacks. However, few methods study the robustness and training efficiency of quantized networks. To this end, we propose a new method by recasting robust quantization as an online domain generalization problem, termed ODG-Q, which generates diverse adversarial data at a low cost during training. ODG-Q consistently outperforms existing works against various adversarial attacks. For example, on CIFAR-10 dataset, ODG-Q achieves 49.2% average improvements under five common white-box attacks and 21.7% average improvements under five common black-box attacks, with a training cost similar to that of natural training (viz. without adversaries). To our best knowledge, this work is the first work that trains both quantized and binary neural networks on ImageNet that consistently improve robustness under different attacks. We also provide a theoretical insight of ODG-Q that accounts for the bound of model risk on attacked data.