Histogram-Equalized Quantization for logic-gated Residual Neural Networks

TL;DR

This paper introduces Histogram-Equalized Quantization (HEQ), an adaptive method for neural network quantization that improves accuracy and efficiency, especially for logic-gated residual networks, by automatically optimizing quantization thresholds.

Contribution

The paper proposes HEQ, a novel adaptive quantization framework that enhances neural network performance and hardware efficiency, particularly for logic-gated residual architectures.

Findings

HEQ achieves state-of-the-art results on CIFAR-10.

HEQ enables effective training of logic-gated residual networks.

HEQ provides higher accuracy at lower hardware complexity.

Abstract

Adjusting the quantization according to the data or to the model loss seems mandatory to enable a high accuracy in the context of quantized neural networks. This work presents Histogram-Equalized Quantization (HEQ), an adaptive framework for linear symmetric quantization. HEQ automatically adapts the quantization thresholds using a unique step size optimization. We empirically show that HEQ achieves state-of-the-art performances on CIFAR-10. Experiments on the STL-10 dataset even show that HEQ enables a proper training of our proposed logic-gated (OR, MUX) residual networks with a higher accuracy at a lower hardware complexity than previous work.