Blending Pruning Criteria for Convolutional Neural Networks

TL;DR

This paper introduces a novel framework that combines multiple filter pruning criteria for CNNs by exploring their diversity, leading to more effective model compression and improved performance on benchmark datasets.

Contribution

The paper proposes a new method to integrate various pruning criteria through clustering and calibration, enhancing CNN pruning effectiveness.

Findings

Outperforms state-of-the-art pruning methods on CIFAR-100 and ImageNet.

Improves model compactness while maintaining or enhancing accuracy.

Demonstrates the benefit of combining multiple pruning criteria for CNNs.

Abstract

The advancement of convolutional neural networks (CNNs) on various vision applications has attracted lots of attention. Yet the majority of CNNs are unable to satisfy the strict requirement for real-world deployment. To overcome this, the recent popular network pruning is an effective method to reduce the redundancy of the models. However, the ranking of filters according to their "importance" on different pruning criteria may be inconsistent. One filter could be important according to a certain criterion, while it is unnecessary according to another one, which indicates that each criterion is only a partial view of the comprehensive "importance". From this motivation, we propose a novel framework to integrate the existing filter pruning criteria by exploring the criteria diversity. The proposed framework contains two stages: Criteria Clustering and Filters Importance Calibration. First, we condense the pruning criteria via layerwise clustering based on the rank of "importance" score. Second, within each cluster, we propose a calibration factor to adjust their significance for each selected blending candidates and search for the optimal blending criterion via Evolutionary Algorithm. Quantitative results on the CIFAR-100 and ImageNet benchmarks show that our framework outperforms the state-of-the-art baselines, regrading to the compact model performance after pruning.