Network Pruning via Resource Reallocation

TL;DR

PEEL is a fast channel pruning method that reallocates resources within a predefined backbone to efficiently produce compact models with minimal cost, demonstrating competitive performance on ImageNet.

Contribution

The paper introduces PEEL, a novel resource reallocation approach for rapid channel pruning that outperforms traditional iterative methods in efficiency and effectiveness.

Findings

PEEL achieves competitive accuracy with state-of-the-art pruning methods.

PEEL significantly reduces pruning time compared to iterative approaches.

PEEL maintains high model performance across various architectures.

Abstract

Channel pruning is broadly recognized as an effective approach to obtain a small compact model through eliminating unimportant channels from a large cumbersome network. Contemporary methods typically perform iterative pruning procedure from the original over-parameterized model, which is both tedious and expensive especially when the pruning is aggressive. In this paper, we propose a simple yet effective channel pruning technique, termed network Pruning via rEsource rEalLocation (PEEL), to quickly produce a desired slim model with negligible cost. Specifically, PEEL first constructs a predefined backbone and then conducts resource reallocation on it to shift parameters from less informative layers to more important layers in one round, thus amplifying the positive effect of these informative layers. To demonstrate the effectiveness of PEEL , we perform extensive experiments on ImageNet with ResNet-18, ResNet-50, MobileNetV2, MobileNetV3-small and EfficientNet-B0. Experimental results show that structures uncovered by PEEL exhibit competitive performance with state-of-the-art pruning algorithms under various pruning settings. Our code is available at https://github.com/cardwing/Codes-for-PEEL.