PQV-Mobile: A Combined Pruning and Quantization Toolkit to Optimize Vision Transformers for Mobile Applications
Kshitij Bhardwaj

TL;DR
PQV-Mobile is a toolkit that combines pruning and quantization techniques to optimize vision transformers for mobile devices, achieving significant latency reduction with minimal accuracy loss.
Contribution
This paper introduces PQV-Mobile, a novel combined pruning and quantization toolkit specifically designed for optimizing vision transformers for mobile deployment.
Findings
Achieves 7.18x latency reduction with 2.24% accuracy loss on DeiT models.
Supports multiple structured pruning methods and quantization levels.
Demonstrates effective trade-offs between latency, memory, and accuracy.
Abstract
While Vision Transformers (ViTs) are extremely effective at computer vision tasks and are replacing convolutional neural networks as the new state-of-the-art, they are complex and memory-intensive models. In order to effectively run these models on resource-constrained mobile/edge systems, there is a need to not only compress these models but also to optimize them and convert them into deployment-friendly formats. To this end, this paper presents a combined pruning and quantization tool, called PQV-Mobile, to optimize vision transformers for mobile applications. The tool is able to support different types of structured pruning based on magnitude importance, Taylor importance, and Hessian importance. It also supports quantization from FP32 to FP16 and int8, targeting different mobile hardware backends. We demonstrate the capabilities of our tool and show important latency-memory-accuracy…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsCCD and CMOS Imaging Sensors · Advanced Neural Network Applications · Advanced Image and Video Retrieval Techniques
MethodsResidual Connection · Layer Normalization · Position-Wise Feed-Forward Layer · Adam · Byte Pair Encoding · Absolute Position Encodings · Feedforward Network · Pruning · Softmax · Linear Layer
