CFSP: An Efficient Structured Pruning Framework for LLMs with Coarse-to-Fine Activation Information

TL;DR

CFSP is a novel structured pruning framework for Large Language Models that efficiently leverages activation information at multiple granularities to achieve high sparsity with minimal performance loss.

Contribution

The paper introduces CFSP, a structured pruning method that uses coarse-to-fine activation importance, enabling efficient pruning with a single forward pass and adaptive fine-tuning.

Findings

CFSP outperforms existing pruning methods across various models and sparsity levels.

The framework achieves high sparsity with minimal accuracy degradation.

Experimental results validate the efficiency and effectiveness of CFSP.

Abstract

The colossal parameters and computational overhead of Large Language Models (LLMs) challenge their real-world applications. Network pruning, which targets unstructured or structured sparsity by removing redundant parameters, has recently been explored for LLM acceleration. Existing LLM pruning works focus on unstructured pruning, which typically requires special hardware support for a practical speed-up. In contrast, structured pruning can reduce latency on general devices. However, it remains a challenge to perform structured pruning efficiently and maintain performance, especially at high sparsity ratios. To this end, we introduce an efficient structured pruning framework named CFSP, which leverages both Coarse (interblock) and Fine-grained (intrablock) activation information as an importance criterion to guide pruning. The pruning is highly efficient, as it only requires one forward pass to compute feature activations. Specifically, we first allocate the sparsity budget across blocks based on their importance and then retain important weights within each block. In addition, we introduce a recovery fine-tuning strategy that adaptively allocates training overhead based on coarse-grained importance to further improve performance. Experimental results demonstrate that CFSP outperforms existing methods on diverse models across various sparsity budgets. Our code will be available at https://github.com/wyxscir/CFSP.