Graph Pruning for Model Compression

TL;DR

This paper introduces GraphPruning, a novel neural network pruning method that models networks as graphs and uses graph convolutional networks to better capture inter-layer redundancies, leading to improved model compression.

Contribution

It proposes a graph-based pruning approach with a graph aggregator and reinforcement learning to optimize sub-network configurations, outperforming previous AutoML pruning methods.

Findings

Achieved state-of-the-art results on ImageNet-2012

Effectively captures inter-layer redundancies

Utilizes graph convolution networks for feature aggregation

Abstract

Previous AutoML pruning works utilized individual layer features to automatically prune filters. We analyze the correlation for two layers from the different blocks which have a short-cut structure. It shows that, in one block, the deeper layer has many redundant filters which can be represented by filters in the former layer. So, it is necessary to take information from other layers into consideration in pruning. In this paper, a novel pruning method, named GraphPruning, is proposed. Any series of the network is viewed as a graph. To automatically aggregate neighboring features for each node, a graph aggregator based on graph convolution networks(GCN) is designed. In the training stage, a PruningNet that is given aggregated node features generates reasonable weights for any size of the sub-network. Subsequently, the best configuration of the Pruned Network is searched by reinforcement learning. Different from previous work, we take the node features from a well-trained graph aggregator instead of the hand-craft features, as the states in reinforcement learning. Compared with other AutoML pruning works, our method has achieved the state-of-the-art under the same conditions on ImageNet-2012.