Data Quality-aware Mixed-precision Quantization via Hybrid Reinforcement Learning

TL;DR

This paper introduces DQMQ, a dynamic, data quality-aware mixed-precision quantization framework that uses hybrid reinforcement learning to adapt bit-widths during training, improving robustness and performance on varied data qualities.

Contribution

The paper proposes a novel end-to-end differentiable framework combining reinforcement learning and supervised training for dynamic mixed-precision quantization based on data quality.

Findings

DQMQ outperforms existing fixed/mixed-precision methods.

It adaptively assigns bit-widths based on input data quality.

Demonstrates robustness across various datasets and networks.

Abstract

Mixed-precision quantization mostly predetermines the model bit-width settings before actual training due to the non-differential bit-width sampling process, obtaining sub-optimal performance. Worse still, the conventional static quality-consistent training setting, i.e., all data is assumed to be of the same quality across training and inference, overlooks data quality changes in real-world applications which may lead to poor robustness of the quantized models. In this paper, we propose a novel Data Quality-aware Mixed-precision Quantization framework, dubbed DQMQ, to dynamically adapt quantization bit-widths to different data qualities. The adaption is based on a bit-width decision policy that can be learned jointly with the quantization training. Concretely, DQMQ is modeled as a hybrid reinforcement learning (RL) task that combines model-based policy optimization with supervised quantization training. By relaxing the discrete bit-width sampling to a continuous probability distribution that is encoded with few learnable parameters, DQMQ is differentiable and can be directly optimized end-to-end with a hybrid optimization target considering both task performance and quantization benefits. Trained on mixed-quality image datasets, DQMQ can implicitly select the most proper bit-width for each layer when facing uneven input qualities. Extensive experiments on various benchmark datasets and networks demonstrate the superiority of DQMQ against existing fixed/mixed-precision quantization methods.