Network Pruning via Transformable Architecture Search
Xuanyi Dong, Yi Yang

TL;DR
This paper introduces a novel neural architecture search-based method for network pruning that dynamically determines the optimal channel and layer sizes, leading to more flexible and efficient pruned networks.
Contribution
It proposes a new approach combining neural architecture search with knowledge transfer to directly optimize pruned network structures without fixed configurations.
Findings
Outperforms traditional pruning methods on CIFAR and ImageNet datasets.
Effectively learns network width and depth through back-propagation of loss.
Demonstrates flexible architecture adaptation improves pruning efficiency.
Abstract
Network pruning reduces the computation costs of an over-parameterized network without performance damage. Prevailing pruning algorithms pre-define the width and depth of the pruned networks, and then transfer parameters from the unpruned network to pruned networks. To break the structure limitation of the pruned networks, we propose to apply neural architecture search to search directly for a network with flexible channel and layer sizes. The number of the channels/layers is learned by minimizing the loss of the pruned networks. The feature map of the pruned network is an aggregation of K feature map fragments (generated by K networks of different sizes), which are sampled based on the probability distribution.The loss can be back-propagated not only to the network weights, but also to the parameterized distribution to explicitly tune the size of the channels/layers. Specifically, we…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Malware Detection Techniques · Software Testing and Debugging Techniques · Software-Defined Networks and 5G
MethodsPruning · Sigmoid Activation · Tanh Activation · Average Pooling · Residual Connection · *Communicated@Fast*How Do I Communicate to Expedia? · 1x1 Convolution · Softmax · Batch Normalization · Long Short-Term Memory
