AdaQAT: Adaptive Bit-Width Quantization-Aware Training

TL;DR

AdaQAT introduces a learning-based, adaptive method for optimizing bit-widths in quantization-aware training, enabling efficient mixed-precision DNN inference with minimal manual tuning.

Contribution

It proposes a novel gradient-based approach to automatically optimize weight and activation bit-widths during training, applicable from scratch or fine-tuning.

Findings

Competitive accuracy on CIFAR-10 and ImageNet datasets.

Effective in both training from scratch and fine-tuning scenarios.

Flexible for mixed-precision uniform quantization.

Abstract

Large-scale deep neural networks (DNNs) have achieved remarkable success in many application scenarios. However, high computational complexity and energy costs of modern DNNs make their deployment on edge devices challenging. Model quantization is a common approach to deal with deployment constraints, but searching for optimized bit-widths can be challenging. In this work, we present Adaptive Bit-Width Quantization Aware Training (AdaQAT), a learning-based method that automatically optimizes weight and activation signal bit-widths during training for more efficient DNN inference. We use relaxed real-valued bit-widths that are updated using a gradient descent rule, but are otherwise discretized for all quantization operations. The result is a simple and flexible QAT approach for mixed-precision uniform quantization problems. Compared to other methods that are generally designed to be run on a pretrained network, AdaQAT works well in both training from scratch and fine-tuning scenarios.Initial results on the CIFAR-10 and ImageNet datasets using ResNet20 and ResNet18 models, respectively, indicate that our method is competitive with other state-of-the-art mixed-precision quantization approaches.