Comprehensive Study on Performance Evaluation and Optimization of Model Compression: Bridging Traditional Deep Learning and Large Language Models

TL;DR

This paper comprehensively evaluates and compares various model compression techniques like quantization and pruning across different deep learning models, including large language models, highlighting their impacts on size, accuracy, and inference time.

Contribution

It provides an extensive analysis of compression methods applied to diverse models, bridging traditional deep learning and large language models with practical performance insights.

Findings

Quantization and pruning significantly reduce model size and inference time.

Compression techniques can impact model accuracy variably depending on the method and model type.

Challenges remain in balancing compression efficiency and maintaining high accuracy.

Abstract

Deep learning models have achieved tremendous success in most of the industries in recent years. The evolution of these models has also led to an increase in the model size and energy requirement, making it difficult to deploy in production on low compute devices. An increase in the number of connected devices around the world warrants compressed models that can be easily deployed at the local devices with low compute capacity and power accessibility. A wide range of solutions have been proposed by different researchers to reduce the size and complexity of such models, prominent among them are, Weight Quantization, Parameter Pruning, Network Pruning, low-rank representation, weights sharing, neural architecture search, knowledge distillation etc. In this research work, we investigate the performance impacts on various trained deep learning models, compressed using quantization and pruning techniques. We implemented both, quantization and pruning, compression techniques on popular deep learning models used in the image classification, object detection, language models and generative models-based problem statements. We also explored performance of various large language models (LLMs) after quantization and low rank adaptation. We used the standard evaluation metrics (model's size, accuracy, and inference time) for all the related problem statements and concluded this paper by discussing the challenges and future work.