EvoP: Robust LLM Inference via Evolutionary Pruning

TL;DR

EvoP is an evolutionary pruning framework that enhances large language model efficiency and robustness by intelligently searching for optimal pruning patterns using a diverse calibration dataset.

Contribution

EvoP introduces a novel evolutionary pruning method and a cluster-based dataset sampling strategy to improve LLM pruning performance and robustness.

Findings

EvoP outperforms existing pruning methods in accuracy and efficiency.

EvoP maintains high performance across various LLMs and tasks.

The framework is practical and scalable for real-world deployment.

Abstract

Large Language Models (LLMs) have achieved remarkable success in natural language processing tasks, but their massive size and computational demands hinder their deployment in resource-constrained environments. Existing model pruning methods address this issue by removing redundant structures (e.g., elements, channels, layers) from the model. However, these methods employ a heuristic pruning strategy, which leads to suboptimal performance. Besides, they also ignore the data characteristics when pruning the model. To overcome these limitations, we propose EvoP, an evolutionary pruning framework for robust LLM inference. EvoP first presents a cluster-based calibration dataset sampling (CCDS) strategy for creating a more diverse calibration dataset. EvoP then introduces an evolutionary pruning pattern searching (EPPS) method to find the optimal pruning pattern. Compared to existing model pruning techniques, EvoP achieves the best performance while maintaining the best efficiency. Experiments across different LLMs and different downstream tasks validate the effectiveness of the proposed EvoP, making it a practical and scalable solution for deploying LLMs in real-world applications.