Class-based Quantization for Neural Networks

TL;DR

This paper introduces a class-based quantization approach for neural networks that assigns different bit-widths to filters or neurons based on their importance, effectively reducing model size and computation while maintaining accuracy.

Contribution

The proposed method evaluates filter importance relative to dataset classes and uses a search algorithm to assign optimal quantization bits per filter or neuron.

Findings

Maintains inference accuracy with low-bit quantization.

Outperforms existing methods at the same quantization levels.

Achieves better accuracy than prior approaches.

Abstract

In deep neural networks (DNNs), there are a huge number of weights and multiply-and-accumulate (MAC) operations. Accordingly, it is challenging to apply DNNs on resource-constrained platforms, e.g., mobile phones. Quantization is a method to reduce the size and the computational complexity of DNNs. Existing quantization methods either require hardware overhead to achieve a non-uniform quantization or focus on model-wise and layer-wise uniform quantization, which are not as fine-grained as filter-wise quantization. In this paper, we propose a class-based quantization method to determine the minimum number of quantization bits for each filter or neuron in DNNs individually. In the proposed method, the importance score of each filter or neuron with respect to the number of classes in the dataset is first evaluated. The larger the score is, the more important the filter or neuron is and thus the larger the number of quantization bits should be. Afterwards, a search algorithm is adopted to exploit the different importance of filters and neurons to determine the number of quantization bits of each filter or neuron. Experimental results demonstrate that the proposed method can maintain the inference accuracy with low bit-width quantization. Given the same number of quantization bits, the proposed method can also achieve a better inference accuracy than the existing methods.