Learning to Prune in Training via Dynamic Channel Propagation

TL;DR

This paper introduces a dynamic channel propagation method that prunes neural networks during training by evaluating channel utility, leading to efficient training and effective network compression.

Contribution

It presents a novel training mechanism that adaptively prunes channels during training based on their utility, improving efficiency and performance.

Findings

Achieves superior performance on benchmark datasets.

Effectively prunes networks during training, reducing complexity.

Demonstrates robustness across CNN architectures.

Abstract

In this paper, we propose a novel network training mechanism called "dynamic channel propagation" to prune the neural networks during the training period. In particular, we pick up a specific group of channels in each convolutional layer to participate in the forward propagation in training time according to the significance level of channel, which is defined as channel utility. The utility values with respect to all selected channels are updated simultaneously with the error back-propagation process and will adaptively change. Furthermore, when the training ends, channels with high utility values are retained whereas those with low utility values are discarded. Hence, our proposed scheme trains and prunes neural networks simultaneously. We empirically evaluate our novel training scheme on various representative benchmark datasets and advanced convolutional neural network (CNN) architectures, including VGGNet and ResNet. The experiment results verify the superior performance and robust effectiveness of our approach.