Hybrid and Non-Uniform quantization methods using retro synthesis data for efficient inference

TL;DR

This paper introduces a data-independent post-training quantization method using Retro-Synthesis Data, achieving superior performance on various models and datasets, and proposes hybrid and non-uniform quantization variants for efficient inference.

Contribution

It presents a novel data-independent quantization scheme that generates faux data from model statistics, outperforming existing methods and introducing new quantization variants.

Findings

Outperformed state-of-the-art methods like ZeroQ and DFQ on ImageNet and CIFAR-10.

Effective for models with and without Batch-Normalization layers.

Applicable across 8, 6, and 4-bit precisions.

Abstract

Existing quantization aware training methods attempt to compensate for the quantization loss by leveraging on training data, like most of the post-training quantization methods, and are also time consuming. Both these methods are not effective for privacy constraint applications as they are tightly coupled with training data. In contrast, this paper proposes a data-independent post-training quantization scheme that eliminates the need for training data. This is achieved by generating a faux dataset, hereafter referred to as Retro-Synthesis Data, from the FP32 model layer statistics and further using it for quantization. This approach outperformed state-of-the-art methods including, but not limited to, ZeroQ and DFQ on models with and without Batch-Normalization layers for 8, 6, and 4 bit precisions on ImageNet and CIFAR-10 datasets. We also introduced two futuristic variants of post-training quantization methods namely Hybrid Quantization and Non-Uniform Quantization