LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks

TL;DR

LQ-Nets introduces a joint training approach for learned quantizers in deep neural networks, significantly improving accuracy of quantized models across various architectures and datasets, while maintaining compatibility with bit-operations.

Contribution

The paper presents a novel method for jointly training quantized DNNs and their quantizers, outperforming fixed schemes and applicable to arbitrary-bit precision.

Findings

Consistently outperforms previous quantization methods in accuracy.

Effective across multiple network architectures and datasets.

Quantizers are easy to train and compatible with bit-operations.

Abstract

Although weight and activation quantization is an effective approach for Deep Neural Network (DNN) compression and has a lot of potentials to increase inference speed leveraging bit-operations, there is still a noticeable gap in terms of prediction accuracy between the quantized model and the full-precision model. To address this gap, we propose to jointly train a quantized, bit-operation-compatible DNN and its associated quantizers, as opposed to using fixed, handcrafted quantization schemes such as uniform or logarithmic quantization. Our method for learning the quantizers applies to both network weights and activations with arbitrary-bit precision, and our quantizers are easy to train. The comprehensive experiments on CIFAR-10 and ImageNet datasets show that our method works consistently well for various network structures such as AlexNet, VGG-Net, GoogLeNet, ResNet, and DenseNet, surpassing previous quantization methods in terms of accuracy by an appreciable margin. Code available at https://github.com/Microsoft/LQ-Nets