EMQ: Evolving Training-free Proxies for Automated Mixed Precision Quantization

TL;DR

This paper introduces EMQ, an automated framework that evolves training-free proxies for mixed-precision quantization, significantly improving search efficiency and accuracy without heavy tuning or expert knowledge.

Contribution

The paper presents an automatic proxy search framework using evolving algorithms to discover highly correlated proxies for mixed-precision quantization.

Findings

EMQ outperforms state-of-the-art methods on ImageNet.

The evolved proxies achieve better correlation with quantization accuracy.

The framework reduces search cost and complexity.

Abstract

Mixed-Precision Quantization~(MQ) can achieve a competitive accuracy-complexity trade-off for models. Conventional training-based search methods require time-consuming candidate training to search optimized per-layer bit-width configurations in MQ. Recently, some training-free approaches have presented various MQ proxies and significantly improve search efficiency. However, the correlation between these proxies and quantization accuracy is poorly understood. To address the gap, we first build the MQ-Bench-101, which involves different bit configurations and quantization results. Then, we observe that the existing training-free proxies perform weak correlations on the MQ-Bench-101. To efficiently seek superior proxies, we develop an automatic search of proxies framework for MQ via evolving algorithms. In particular, we devise an elaborate search space involving the existing proxies and perform an evolution search to discover the best correlated MQ proxy. We proposed a diversity-prompting selection strategy and compatibility screening protocol to avoid premature convergence and improve search efficiency. In this way, our Evolving proxies for Mixed-precision Quantization~(EMQ) framework allows the auto-generation of proxies without heavy tuning and expert knowledge. Extensive experiments on ImageNet with various ResNet and MobileNet families demonstrate that our EMQ obtains superior performance than state-of-the-art mixed-precision methods at a significantly reduced cost. The code will be released.