WHC: Weighted Hybrid Criterion for Filter Pruning on Convolutional Neural Networks

TL;DR

This paper introduces WHC, a new filter pruning criterion for CNNs that considers filter magnitude and dependence, enabling more effective pruning without performance loss.

Contribution

The paper proposes a novel data-independent filter pruning criterion, WHC, addressing limitations of existing methods by incorporating filter dissimilarity measures.

Findings

WHC prunes ResNet-50 with over 42% FLOPs reduction on ImageNet.

WHC maintains top-5 accuracy after pruning.

Extensive experiments validate the effectiveness of WHC.

Abstract

Filter pruning has attracted increasing attention in recent years for its capacity in compressing and accelerating convolutional neural networks. Various data-independent criteria, including norm-based and relationship-based ones, were proposed to prune the most unimportant filters. However, these state-of-the-art criteria fail to fully consider the dissimilarity of filters, and thus might lead to performance degradation. In this paper, we first analyze the limitation of relationship-based criteria with examples, and then introduce a new data-independent criterion, Weighted Hybrid Criterion (WHC), to tackle the problems of both norm-based and relationship-based criteria. By taking the magnitude of each filter and the linear dependence between filters into consideration, WHC can robustly recognize the most redundant filters, which can be safely pruned without introducing severe performance degradation to networks. Extensive pruning experiments in a simple one-shot manner demonstrate the effectiveness of the proposed WHC. In particular, WHC can prune ResNet-50 on ImageNet with more than 42% of floating point operations reduced without any performance loss in top-5 accuracy.