MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning

TL;DR

MetaPruning introduces a meta learning framework with a PruningNet that efficiently generates weights for pruned networks, enabling automatic, constraint-aware neural network pruning without finetuning.

Contribution

The paper presents a novel meta learning approach with a PruningNet for efficient, automatic channel pruning of deep neural networks, reducing human effort and computational cost.

Findings

Outperforms state-of-the-art pruning methods on MobileNet and ResNet.

Enables rapid search for various pruned networks under different constraints.

Eliminates the need for finetuning during the search process.

Abstract

In this paper, we propose a novel meta learning approach for automatic channel pruning of very deep neural networks. We first train a PruningNet, a kind of meta network, which is able to generate weight parameters for any pruned structure given the target network. We use a simple stochastic structure sampling method for training the PruningNet. Then, we apply an evolutionary procedure to search for good-performing pruned networks. The search is highly efficient because the weights are directly generated by the trained PruningNet and we do not need any finetuning at search time. With a single PruningNet trained for the target network, we can search for various Pruned Networks under different constraints with little human participation. Compared to the state-of-the-art pruning methods, we have demonstrated superior performances on MobileNet V1/V2 and ResNet. Codes are available on https://github.com/liuzechun/MetaPruning.