Efficient Multi-bit Quantization Network Training via Weight Bias Correction and Bit-wise Coreset Sampling

TL;DR

This paper introduces techniques to efficiently train multi-bit quantization networks by correcting weight biases and sampling informative data subsets, significantly reducing training time while maintaining high accuracy.

Contribution

The authors propose weight bias correction and bit-wise coreset sampling methods to cut training overhead in multi-bit quantization networks without sacrificing performance.

Findings

Achieves up to 7.88x reduction in training time.

Maintains or improves accuracy across multiple datasets and architectures.

Eliminates the need for fine-tuning for different bit-widths.

Abstract

Multi-bit quantization networks enable flexible deployment of deep neural networks by supporting multiple precision levels within a single model. However, existing approaches suffer from significant training overhead as full-dataset updates are repeated for each supported bit-width, resulting in a cost that scales linearly with the number of precisions. Additionally, extra fine-tuning stages are often required to support additional or intermediate precision options, further compounding the overall training burden. To address this issue, we propose two techniques that greatly reduce the training overhead without compromising model utility: (i) Weight bias correction enables shared batch normalization and eliminates the need for fine-tuning by neutralizing quantization-induced bias across bit-widths and aligning activation distributions; and (ii) Bit-wise coreset sampling strategy allows each child model to train on a compact, informative subset selected via gradient-based importance scores by exploiting the implicit knowledge transfer phenomenon. Experiments on CIFAR-10/100, TinyImageNet, and ImageNet-1K with both ResNet and ViT architectures demonstrate that our method achieves competitive or superior accuracy while reducing training time up to 7.88x. Our code is released at https://github.com/a2jinhee/EMQNet_jk.