Balanced Quantization: An Effective and Efficient Approach to Quantized Neural Networks

TL;DR

This paper introduces a balanced quantization method for neural networks that improves accuracy by addressing distribution imbalances in parameters, without increasing inference computation or training time.

Contribution

It proposes a novel percentile-based recursive partitioning approach for balanced quantization, enhancing QNN performance on standard datasets.

Findings

Improved top-5 error rate on ImageNet with 4-bit quantized GoogLeNet

Effective for both CNNs and RNNs without extra inference cost

Outperforms state-of-the-art QNN methods

Abstract

Quantized Neural Networks (QNNs), which use low bitwidth numbers for representing parameters and performing computations, have been proposed to reduce the computation complexity, storage size and memory usage. In QNNs, parameters and activations are uniformly quantized, such that the multiplications and additions can be accelerated by bitwise operations. However, distributions of parameters in Neural Networks are often imbalanced, such that the uniform quantization determined from extremal values may under utilize available bitwidth. In this paper, we propose a novel quantization method that can ensure the balance of distributions of quantized values. Our method first recursively partitions the parameters by percentiles into balanced bins, and then applies uniform quantization. We also introduce computationally cheaper approximations of percentiles to reduce the computation overhead introduced. Overall, our method improves the prediction accuracies of QNNs without introducing extra computation during inference, has negligible impact on training speed, and is applicable to both Convolutional Neural Networks and Recurrent Neural Networks. Experiments on standard datasets including ImageNet and Penn Treebank confirm the effectiveness of our method. On ImageNet, the top-5 error rate of our 4-bit quantized GoogLeNet model is 12.7\%, which is superior to the state-of-the-arts of QNNs.