Long-Range Zero-Shot Generative Deep Network Quantization

TL;DR

This paper introduces LRQ, a novel zero-shot quantization method using a long-range generator with attention and adversarial margin techniques, significantly improving the quality of synthetic data and quantization performance without real data.

Contribution

The paper proposes a long-range generator with attention and an adversarial margin module, enhancing zero-shot quantization by producing more global and diverse synthetic data.

Findings

LRQ outperforms existing zero-shot quantization methods.

Long-range attention improves synthetic data diversity.

Decoupled knowledge distillation enhances transfer from full-precision models.

Abstract

Quantization approximates a deep network model with floating-point numbers by the one with low bit width numbers, in order to accelerate inference and reduce computation. Quantizing a model without access to the original data, zero-shot quantization can be accomplished by fitting the real data distribution by data synthesis. However, zero-shot quantization achieves inferior performance compared to the post-training quantization with real data. We find it is because: 1) a normal generator is hard to obtain high diversity of synthetic data, since it lacks long-range information to allocate attention to global features; 2) the synthetic images aim to simulate the statistics of real data, which leads to weak intra-class heterogeneity and limited feature richness. To overcome these problems, we propose a novel deep network quantizer, dubbed Long-Range Zero-Shot Generative Deep Network Quantization (LRQ). Technically, we propose a long-range generator to learn long-range information instead of simple local features. In order for the synthetic data to contain more global features, long-range attention using large kernel convolution is incorporated into the generator. In addition, we also present an Adversarial Margin Add (AMA) module to force intra-class angular enlargement between feature vector and class center. As AMA increases the convergence difficulty of the loss function, which is opposite to the training objective of the original loss function, it forms an adversarial process. Furthermore, in order to transfer knowledge from the full-precision network, we also utilize a decoupled knowledge distillation. Extensive experiments demonstrate that LRQ obtains better performance than other competitors.