MGRQ: Post-Training Quantization For Vision Transformer With Mixed Granularity Reconstruction

TL;DR

This paper introduces MGRQ, a novel mixed granularity reconstruction method for post-training quantization of Vision Transformers, significantly improving accuracy especially in low-bit scenarios by employing global and local supervision strategies.

Contribution

MGRQ is the first to apply mixed granularity reconstruction with global and local supervision to enhance PTQ performance in Vision Transformers.

Findings

MGRQ outperforms existing methods in low-bit quantization accuracy.

Global and local supervision strategies effectively reduce quantization errors.

Extensive experiments validate the robustness and practicality of MGRQ.

Abstract

Post-training quantization (PTQ) efficiently compresses vision models, but unfortunately, it accompanies a certain degree of accuracy degradation. Reconstruction methods aim to enhance model performance by narrowing the gap between the quantized model and the full-precision model, often yielding promising results. However, efforts to significantly improve the performance of PTQ through reconstruction in the Vision Transformer (ViT) have shown limited efficacy. In this paper, we conduct a thorough analysis of the reasons for this limited effectiveness and propose MGRQ (Mixed Granularity Reconstruction Quantization) as a solution to address this issue. Unlike previous reconstruction schemes, MGRQ introduces a mixed granularity reconstruction approach. Specifically, MGRQ enhances the performance of PTQ by introducing Extra-Block Global Supervision and Intra-Block Local Supervision, building upon Optimized Block-wise Reconstruction. Extra-Block Global Supervision considers the relationship between block outputs and the model's output, aiding block-wise reconstruction through global supervision. Meanwhile, Intra-Block Local Supervision reduces generalization errors by aligning the distribution of outputs at each layer within a block. Subsequently, MGRQ is further optimized for reconstruction through Mixed Granularity Loss Fusion. Extensive experiments conducted on various ViT models illustrate the effectiveness of MGRQ. Notably, MGRQ demonstrates robust performance in low-bit quantization, thereby enhancing the practicality of the quantized model.