Zero-Shot Learning of a Conditional Generative Adversarial Network for Data-Free Network Quantization

TL;DR

This paper introduces ZS-CGAN, a zero-shot conditional GAN that generates synthetic data for data-free neural network quantization, achieving state-of-the-art results with minimal accuracy loss.

Contribution

The paper presents a novel zero-shot training method for CGANs that does not require training data, enabling effective data-free network quantization.

Findings

Achieved state-of-the-art data-free quantization results on ResNet and MobileNet.

Minimal accuracy loss compared to traditional data-dependent quantization.

Demonstrated effectiveness of ZS-CGAN in practical neural network compression.

Abstract

We propose a novel method for training a conditional generative adversarial network (CGAN) without the use of training data, called zero-shot learning of a CGAN (ZS-CGAN). Zero-shot learning of a conditional generator only needs a pre-trained discriminative (classification) model and does not need any training data. In particular, the conditional generator is trained to produce labeled synthetic samples whose characteristics mimic the original training data by using the statistics stored in the batch normalization layers of the pre-trained model. We show the usefulness of ZS-CGAN in data-free quantization of deep neural networks. We achieved the state-of-the-art data-free network quantization of the ResNet and MobileNet classification models trained on the ImageNet dataset. Data-free quantization using ZS-CGAN showed a minimal loss in accuracy compared to that obtained by conventional data-dependent quantization.