CSQ: Growing Mixed-Precision Quantization Scheme with Bi-level Continuous Sparsification

TL;DR

CSQ introduces a stable, fully-differentiable bi-level continuous sparsification method for mixed-precision quantization in DNNs, enabling efficient search for optimal precision schemes with improved accuracy-efficiency tradeoffs.

Contribution

The paper proposes CSQ, a novel bi-level continuous sparsification approach for stable, differentiable mixed-precision quantization scheme search in neural networks.

Findings

CSQ outperforms previous methods in efficiency-accuracy tradeoff.

It enables dynamic growth and pruning of layer precisions.

Experiments validate improved stability and performance across models and datasets.

Abstract

Mixed-precision quantization has been widely applied on deep neural networks (DNNs) as it leads to significantly better efficiency-accuracy tradeoffs compared to uniform quantization. Meanwhile, determining the exact precision of each layer remains challenging. Previous attempts on bit-level regularization and pruning-based dynamic precision adjustment during training suffer from noisy gradients and unstable convergence. In this work, we propose Continuous Sparsification Quantization (CSQ), a bit-level training method to search for mixed-precision quantization schemes with improved stability. CSQ stabilizes the bit-level mixed-precision training process with a bi-level gradual continuous sparsification on both the bit values of the quantized weights and the bit selection in determining the quantization precision of each layer. The continuous sparsification scheme enables fully-differentiable training without gradient approximation while achieving an exact quantized model in the end.A budget-aware regularization of total model size enables the dynamic growth and pruning of each layer's precision towards a mixed-precision quantization scheme of the desired size. Extensive experiments show CSQ achieves better efficiency-accuracy tradeoff than previous methods on multiple models and datasets.