Channel Pruning via Multi-Criteria based on Weight Dependency

TL;DR

This paper introduces CPMC, a multi-criteria channel pruning method that considers weight dependency and multiple importance factors to efficiently compress CNN models like VGGNet, ResNet, and DenseNet.

Contribution

It proposes a novel channel importance evaluation based on weight dependency and multiple criteria, improving pruning effectiveness and efficiency.

Findings

CPMC outperforms existing pruning methods significantly.

It effectively compresses various CNN architectures.

The method maintains high accuracy after pruning.

Abstract

Channel pruning has demonstrated its effectiveness in compressing ConvNets. In many related arts, the importance of an output feature map is only determined by its associated filter. However, these methods ignore a small part of weights in the next layer which disappears as the feature map is removed. They ignore the phenomenon of weight dependency. Besides, many pruning methods use only one criterion for evaluation and find a sweet spot of pruning structure and accuracy in a trial-and-error fashion, which can be time-consuming. In this paper, we proposed a channel pruning algorithm via multi-criteria based on weight dependency, CPMC, which can compress a pre-trained model directly. CPMC defines channel importance in three aspects, including its associated weight value, computational cost, and parameter quantity. According to the phenomenon of weight dependency, CPMC gets channel importance by assessing its associated filter and the corresponding partial weights in the next layer. Then CPMC uses global normalization to achieve cross-layer comparison. Finally, CPMC removes less important channels by global ranking. CPMC can compress various CNN models, including VGGNet, ResNet, and DenseNet on various image classification datasets. Extensive experiments have shown CPMC outperforms the others significantly.