EfficientQuant: An Efficient Post-Training Quantization for CNN-Transformer Hybrid Models on Edge Devices

TL;DR

EfficientQuant introduces a structure-aware post-training quantization method that significantly reduces latency and resource usage of hybrid CNN-Transformer models on edge devices with minimal accuracy loss.

Contribution

It presents a novel PTQ approach tailored for hybrid models, applying different quantization schemes to convolutional and transformer blocks for improved efficiency.

Findings

Achieves 2.5x to 8.7x latency reduction on ImageNet-1K.

Maintains high accuracy with minimal loss.

Demonstrates practicality on edge devices.

Abstract

Hybrid models that combine convolutional and transformer blocks offer strong performance in computer vision (CV) tasks but are resource-intensive for edge deployment. Although post-training quantization (PTQ) can help reduce resource demand, its application to hybrid models remains limited. We propose EfficientQuant, a novel structure-aware PTQ approach that applies uniform quantization to convolutional blocks and $log_2$ quantization to transformer blocks. EfficientQuant achieves $2.5 \times - 8.7 \times$ latency reduction with minimal accuracy loss on the ImageNet-1K dataset. It further demonstrates low latency and memory efficiency on edge devices, making it practical for real-world deployment.