Network Pruning via Feature Shift Minimization

TL;DR

This paper introduces a novel feature shift minimization approach for channel pruning in CNNs, which evaluates and minimizes feature shifts to improve compression efficiency and accuracy, outperforming existing methods.

Contribution

The paper proposes a new feature shift concept and an approximation method for efficient CNN model compression, along with a distribution-optimization algorithm to enhance accuracy.

Findings

Achieves state-of-the-art performance on benchmark networks

Effectively reduces model complexity while maintaining accuracy

Demonstrates robustness across various datasets

Abstract

Channel pruning is widely used to reduce the complexity of deep network models. Recent pruning methods usually identify which parts of the network to discard by proposing a channel importance criterion. However, recent studies have shown that these criteria do not work well in all conditions. In this paper, we propose a novel Feature Shift Minimization (FSM) method to compress CNN models, which evaluates the feature shift by converging the information of both features and filters. Specifically, we first investigate the compression efficiency with some prevalent methods in different layer-depths and then propose the feature shift concept. Then, we introduce an approximation method to estimate the magnitude of the feature shift, since it is difficult to compute it directly. Besides, we present a distribution-optimization algorithm to compensate for the accuracy loss and improve the network compression efficiency. The proposed method yields state-of-the-art performance on various benchmark networks and datasets, verified by extensive experiments. Our codes are available at: https://github.com/lscgx/FSM.