DivQAT: Enhancing Robustness of Quantized Convolutional Neural Networks against Model Extraction Attacks

TL;DR

DivQAT introduces a novel quantization-aware training method that enhances the robustness of quantized CNNs against model extraction attacks, integrating defense mechanisms directly into the training process for improved security without sacrificing accuracy.

Contribution

This paper presents the first quantization process modification to incorporate model extraction defenses during training, improving robustness of quantized CNNs.

Findings

Effective defense against extraction attacks demonstrated on benchmark datasets.

Maintains model accuracy while enhancing robustness.

Combining DivQAT with other defenses yields superior protection.

Abstract

Convolutional Neural Networks (CNNs) and their quantized counterparts are vulnerable to extraction attacks, posing a significant threat of IP theft. Yet, the robustness of quantized models against these attacks is little studied compared to large models. Previous defenses propose to inject calculated noise into the prediction probabilities. However, these defenses are limited since they are not incorporated during the model design and are only added as an afterthought after training. Additionally, most defense techniques are computationally expensive and often have unrealistic assumptions about the victim model that are not feasible in edge device implementations and do not apply to quantized models. In this paper, we propose DivQAT, a novel algorithm to train quantized CNNs based on Quantization Aware Training (QAT) aiming to enhance their robustness against extraction attacks. To the best of our knowledge, our technique is the first to modify the quantization process to integrate a model extraction defense into the training process. Through empirical validation on benchmark vision datasets, we demonstrate the efficacy of our technique in defending against model extraction attacks without compromising model accuracy. Furthermore, combining our quantization technique with other defense mechanisms improves their effectiveness compared to traditional QAT.