Generative Zero-shot Network Quantization

TL;DR

This paper introduces a zero-shot network quantization method that generates realistic images using Batch Normalization statistics, enabling effective model quantization without access to original training data.

Contribution

It proposes a novel data-free quantization approach leveraging BN statistics to generate calibration data, improving quantization performance without training data.

Findings

Outperforms existing data-free quantization methods on benchmark datasets

Generates realistic images for calibration without any training data

Effectively preserves model accuracy in privacy-sensitive scenarios

Abstract

Convolutional neural networks are able to learn realistic image priors from numerous training samples in low-level image generation and restoration. We show that, for high-level image recognition tasks, we can further reconstruct "realistic" images of each category by leveraging intrinsic Batch Normalization (BN) statistics without any training data. Inspired by the popular VAE/GAN methods, we regard the zero-shot optimization process of synthetic images as generative modeling to match the distribution of BN statistics. The generated images serve as a calibration set for the following zero-shot network quantizations. Our method meets the needs for quantizing models based on sensitive information, \textit{e.g.,} due to privacy concerns, no data is available. Extensive experiments on benchmark datasets show that, with the help of generated data, our approach consistently outperforms existing data-free quantization methods.