PQV-Mobile: A Combined Pruning and Quantization Toolkit to Optimize Vision Transformers for Mobile Applications

TL;DR

PQV-Mobile is a toolkit that combines pruning and quantization techniques to optimize vision transformers for mobile devices, achieving significant latency reduction with minimal accuracy loss.

Contribution

This paper introduces PQV-Mobile, a novel combined pruning and quantization toolkit specifically designed for optimizing vision transformers for mobile deployment.

Findings

Achieves 7.18x latency reduction with 2.24% accuracy loss on DeiT models.

Supports multiple structured pruning methods and quantization levels.

Demonstrates effective trade-offs between latency, memory, and accuracy.

Abstract

While Vision Transformers (ViTs) are extremely effective at computer vision tasks and are replacing convolutional neural networks as the new state-of-the-art, they are complex and memory-intensive models. In order to effectively run these models on resource-constrained mobile/edge systems, there is a need to not only compress these models but also to optimize them and convert them into deployment-friendly formats. To this end, this paper presents a combined pruning and quantization tool, called PQV-Mobile, to optimize vision transformers for mobile applications. The tool is able to support different types of structured pruning based on magnitude importance, Taylor importance, and Hessian importance. It also supports quantization from FP32 to FP16 and int8, targeting different mobile hardware backends. We demonstrate the capabilities of our tool and show important latency-memory-accuracy trade-offs for different amounts of pruning and int8 quantization with Facebook Data Efficient Image Transformer (DeiT) models. Our results show that even pruning a DeiT model by 9.375% and quantizing it to int8 from FP32 followed by optimizing for mobile applications, we find a latency reduction by 7.18X with a small accuracy loss of 2.24%. The tool is open source.