Transform Quantization for CNN (Convolutional Neural Network) Compression

TL;DR

This paper introduces a transform quantization method for CNN weight compression that optimally decorrelates and quantizes weights post-training, improving compression efficiency and performance at low bit-rates.

Contribution

It develops a rate-distortion framework and an optimal end-to-end learned transform for CNN weight quantization, unifying decorrelation and quantization in a single approach.

Findings

Achieves state-of-the-art CNN compression at 1-2 bits.

Effective in both retrained and non-retrained scenarios.

Improves compression without significant loss of accuracy.

Abstract

In this paper, we compress convolutional neural network (CNN) weights post-training via transform quantization. Previous CNN quantization techniques tend to ignore the joint statistics of weights and activations, producing sub-optimal CNN performance at a given quantization bit-rate, or consider their joint statistics during training only and do not facilitate efficient compression of already trained CNN models. We optimally transform (decorrelate) and quantize the weights post-training using a rate-distortion framework to improve compression at any given quantization bit-rate. Transform quantization unifies quantization and dimensionality reduction (decorrelation) techniques in a single framework to facilitate low bit-rate compression of CNNs and efficient inference in the transform domain. We first introduce a theory of rate and distortion for CNN quantization, and pose optimum quantization as a rate-distortion optimization problem. We then show that this problem can be solved using optimal bit-depth allocation following decorrelation by the optimal End-to-end Learned Transform (ELT) we derive in this paper. Experiments demonstrate that transform quantization advances the state of the art in CNN compression in both retrained and non-retrained quantization scenarios. In particular, we find that transform quantization with retraining is able to compress CNN models such as AlexNet, ResNet and DenseNet to very low bit-rates (1-2 bits).