IG-Pruning: Input-Guided Block Pruning for Large Language Models

TL;DR

IG-Pruning introduces an input-aware, dynamic block pruning method for large language models, significantly improving inference efficiency and performance over static methods without extensive retraining.

Contribution

It presents a novel input-guided dynamic pruning approach that adaptively selects transformer layers at inference time, enhancing efficiency for resource-limited deployments.

Findings

Outperforms static depth pruning methods in various tasks

Enables dynamic, input-aware layer selection during inference

Reduces computational costs without extensive retraining

Abstract

With the growing computational demands of large language models (LLMs), efficient inference has become increasingly critical for practical deployment. Depth pruning has emerged as a promising approach for reducing the computational costs of large language models by removing transformer layers. However, existing methods typically rely on fixed block masks, which can lead to suboptimal performance across different tasks and inputs. In this paper, we propose IG-Pruning, a novel input-aware block-wise pruning method that dynamically selects layer masks at inference time. Our approach consists of two stages: (1) Discovering diverse mask candidates through semantic clustering and L0 optimization, and (2) Implementing efficient dynamic pruning without the need for extensive training. Experimental results demonstrate that our method consistently outperforms state-of-the-art static depth pruning methods, making it particularly suitable for resource-constrained deployment scenarios.