Data-Free Quantization with Accurate Activation Clipping and Adaptive Batch Normalization

TL;DR

This paper introduces a data-free quantization method that enhances neural network compression by accurately clipping activations and adaptively updating batch normalization, significantly improving performance without original training data.

Contribution

The paper proposes a novel data-free quantization approach with accurate activation clipping and adaptive batch normalization to reduce quantization errors and improve accuracy.

Findings

Achieves 64.33% top-1 accuracy on ResNet18 with 3.7% improvement.

Outperforms existing state-of-the-art data-free quantization methods.

Demonstrates effectiveness across extensive experiments on ImageNet.

Abstract

Data-free quantization is a task that compresses the neural network to low bit-width without access to original training data. Most existing data-free quantization methods cause severe performance degradation due to inaccurate activation clipping range and quantization error, especially for low bit-width. In this paper, we present a simple yet effective data-free quantization method with accurate activation clipping and adaptive batch normalization. Accurate activation clipping (AAC) improves the model accuracy by exploiting accurate activation information from the full-precision model. Adaptive batch normalization firstly proposes to address the quantization error from distribution changes by updating the batch normalization layer adaptively. Extensive experiments demonstrate that the proposed data-free quantization method can yield surprisingly performance, achieving 64.33% top-1 accuracy of ResNet18 on ImageNet dataset, with 3.7% absolute improvement outperforming the existing state-of-the-art methods.