UWC: Unit-wise Calibration Towards Rapid Network Compression

TL;DR

This paper proposes a unit-wise calibration method for post-training quantization of CNNs, leveraging layer interactions to improve accuracy at very low bit-widths, achieving near-original performance on ImageNet and COCO.

Contribution

It introduces a novel unit-wise feature reconstruction algorithm based on second order Taylor expansion, enhancing quantization accuracy by considering adjacent layer interactions.

Findings

Achieves near-original accuracy on ImageNet with INT4 and INT3 quantization.

Outperforms layer-wise calibration methods in extremely low-bit scenarios.

Demonstrates effectiveness on COCO object detection benchmarks.

Abstract

This paper introduces a post-training quantization~(PTQ) method achieving highly efficient Convolutional Neural Network~ (CNN) quantization with high performance. Previous PTQ methods usually reduce compression error via performing layer-by-layer parameters calibration. However, with lower representational ability of extremely compressed parameters (e.g., the bit-width goes less than 4), it is hard to eliminate all the layer-wise errors. This work addresses this issue via proposing a unit-wise feature reconstruction algorithm based on an observation of second order Taylor series expansion of the unit-wise error. It indicates that leveraging the interaction between adjacent layers' parameters could compensate layer-wise errors better. In this paper, we define several adjacent layers as a Basic-Unit, and present a unit-wise post-training algorithm which can minimize quantization error. This method achieves near-original accuracy on ImageNet and COCO when quantizing FP32 models to INT4 and INT3.