Adaptive Distribution-aware Quantization for Mixed-Precision Neural Networks

TL;DR

This paper introduces ADQ, a novel mixed-precision quantization framework that dynamically adapts codebooks and allocates precision based on distributional insights, significantly improving neural network deployment on resource-limited devices.

Contribution

The paper presents a new adaptive weight quantization scheme with quantile-based initialization, online codebook adaptation, and sensitivity-aware mixed-precision allocation, addressing distribution and static codebook issues in QAT.

Findings

Achieves 71.512% Top-1 accuracy on ImageNet with 2.81 bits for ResNet-18.

Outperforms state-of-the-art methods under similar resource constraints.

Ablation studies confirm the effectiveness of each proposed component.

Abstract

Quantization-Aware Training (QAT) is a critical technique for deploying deep neural networks on resource-constrained devices. However, existing methods often face two major challenges: the highly non-uniform distribution of activations and the static, mismatched codebooks used in weight quantization. To address these challenges, we propose Adaptive Distribution-aware Quantization (ADQ), a mixed-precision quantization framework that employs a differentiated strategy. The core of ADQ is a novel adaptive weight quantization scheme comprising three key innovations: (1) a quantile-based initialization method that constructs a codebook closely aligned with the initial weight distribution; (2) an online codebook adaptation mechanism based on Exponential Moving Average (EMA) to dynamically track distributional shifts; and (3) a sensitivity-informed strategy for mixed-precision allocation. For activations, we integrate a hardware-friendly non-uniform-to-uniform mapping scheme. Comprehensive experiments validate the effectiveness of our method. On ImageNet, ADQ enables a ResNet-18 to achieve 71.512% Top-1 accuracy with an average bit-width of only 2.81 bits, outperforming state-of-the-art methods under comparable conditions. Furthermore, detailed ablation studies on CIFAR-10 systematically demonstrate the individual contributions of each innovative component, validating the rationale and effectiveness of our design.