Where and How to Enhance: Discovering Bit-Width Contribution for Mixed Precision Quantization

TL;DR

This paper introduces a Shapley-based method for mixed precision quantization that more accurately measures the contribution of each bit-width to neural network performance, outperforming traditional gradient-based approaches.

Contribution

It proposes a novel Shapley value approach for bit-width contribution measurement in MPQ, addressing limitations of gradient-based methods.

Findings

SMPQ achieves state-of-the-art results on benchmarks.

Shapley-based contribution measurement improves quantization policy accuracy.

Monte Carlo sampling reduces computation cost in Shapley calculations.

Abstract

Mixed precision quantization (MPQ) is an effective quantization approach to achieve accuracy-complexity trade-off of neural network, through assigning different bit-widths to network activations and weights in each layer. The typical way of existing MPQ methods is to optimize quantization policies (i.e., bit-width allocation) in a gradient descent manner, termed as Differentiable (DMPQ). At the end of the search, the bit-width associated to the quantization parameters which has the largest value will be selected to form the final mixed precision quantization policy, with the implicit assumption that the values of quantization parameters reflect the operation contribution to the accuracy improvement. While much has been discussed about the MPQ improvement, the bit-width selection process has received little attention. We study this problem and argue that the magnitude of quantization parameters does not necessarily reflect the actual contribution of the bit-width to the task performance. Then, we propose a Shapley-based MPQ (SMPQ) method, which measures the bit-width operation direct contribution on the MPQ task. To reduce computation cost, a Monte Carlo sampling-based approximation strategy is proposed for Shapley computation. Extensive experiments on mainstream benchmarks demonstrate that our SMPQ consistently achieves state-of-the-art performance than gradient-based competitors.