Adaptive Binary-Ternary Quantization

TL;DR

This paper introduces Smart Quantization, an adaptive method combining binary and ternary quantization in neural networks, enabling single training with high accuracy on resource-constrained devices.

Contribution

The paper proposes a novel adaptive quantization method that adjusts quantization depth during training, reducing the need for multiple training runs.

Findings

Successfully adapts quantization depth during training

Maintains high accuracy on MNIST and CIFAR10

Reduces training complexity for quantized models

Abstract

Neural network models are resource hungry. It is difficult to deploy such deep networks on devices with limited resources, like smart wearables, cellphones, drones, and autonomous vehicles. Low bit quantization such as binary and ternary quantization is a common approach to alleviate this resource requirements. Ternary quantization provides a more flexible model and outperforms binary quantization in terms of accuracy, however doubles the memory footprint and increases the computational cost. Contrary to these approaches, mixed quantized models allow a trade-off between accuracy and memory footprint. In such models, quantization depth is often chosen manually, or is tuned using a separate optimization routine. The latter requires training a quantized network multiple times. Here, we propose an adaptive combination of binary and ternary quantization, namely Smart Quantization (SQ), in which the quantization depth is modified directly via a regularization function, so that the model is trained only once. Our experimental results show that the proposed method adapts quantization depth successfully while keeping the model accuracy high on MNIST and CIFAR10 benchmarks.