Constraint Guided Model Quantization of Neural Networks

TL;DR

This paper introduces CGMQ, a novel quantization aware training method that reduces neural network complexity for edge deployment by using resource constraints without hyperparameter tuning.

Contribution

CGMQ is a new quantization method that guarantees computational resource limits without hyperparameter tuning, unlike previous approaches.

Findings

Competitive performance on MNIST and CIFAR10

Guarantees upper bound on computational complexity

Does not require hyperparameter tuning

Abstract

Deploying neural networks on the edge has become increasingly important as deep learning is being applied in an increasing amount of applications. At the edge computing hardware typically has limited resources disallowing to run neural networks with high complexity. To reduce the complexity of neural networks a wide range of quantization methods have been proposed in recent years. This work proposes Constraint Guided Model Quantization (CGMQ), which is a quantization aware training algorithm that uses an upper bound on the computational resources and reduces the bit-widths of the parameters of the neural network. CGMQ does not require the tuning of a hyperparameter to result in a mixed precision neural network that satisfies the predefined computational cost constraint, while prior work does. It is shown on MNIST and CIFAR10 that the performance of CGMQ is competitive with state-of-the-art quantization aware training algorithms, while guaranteeing the satisfaction of an upper bound on the computational complexity defined by the computational resources of the on edge hardware.