Mixed-Precision Quantization for Deep Vision Models with Integer Quadratic Programming

TL;DR

This paper introduces CLADO, a novel mixed-precision quantization algorithm for deep vision models that considers cross-layer dependencies to optimize accuracy and efficiency, outperforming existing methods.

Contribution

The paper proposes CLADO, a sensitivity-based MPQ algorithm that captures cross-layer error dependencies using linear approximations and integer quadratic programming.

Findings

CLADO achieves state-of-the-art performance on ImageNet.

It effectively models cross-layer quantization errors.

Results show improved accuracy-efficiency trade-offs.

Abstract

Quantization is a widely used technique to compress neural networks. Assigning uniform bit-widths across all layers can result in significant accuracy degradation at low precision and inefficiency at high precision. Mixed-precision quantization (MPQ) addresses this by assigning varied bit-widths to layers, optimizing the accuracy-efficiency trade-off. Existing sensitivity-based methods for MPQ assume that quantization errors across layers are independent, which leads to suboptimal choices. We introduce CLADO, a practical sensitivity-based MPQ algorithm that captures cross-layer dependency of quantization error. CLADO approximates pairwise cross-layer errors using linear equations on a small data subset. Layerwise bit-widths are assigned by optimizing a new MPQ formulation based on cross-layer quantization errors using an Integer Quadratic Program. Experiments with CNN and vision transformer models on ImageNet demonstrate that CLADO achieves state-of-the-art mixed-precision quantization performance. Code repository available here: https://github.com/JamesTuna/CLADO_MPQ