Dependency Aware Filter Pruning
Kai Zhao, Xin-Yu Zhang, Qi Han, and Ming-Ming Cheng

TL;DR
This paper introduces a dependency-aware filter pruning method for CNNs that considers layer dependencies and dynamically controls sparsity, leading to more efficient network compression.
Contribution
It develops a norm-based importance estimation that accounts for layer dependencies and proposes a dynamic regularization mechanism for better filter pruning.
Findings
Outperforms existing methods on CIFAR, SVHN, and ImageNet datasets.
Effectively identifies unimportant filters considering layer dependencies.
Achieves desired sparsity with improved accuracy and efficiency.
Abstract
Convolutional neural networks (CNNs) are typically over-parameterized, bringing considerable computational overhead and memory footprint in inference. Pruning a proportion of unimportant filters is an efficient way to mitigate the inference cost. For this purpose, identifying unimportant convolutional filters is the key to effective filter pruning. Previous work prunes filters according to either their weight norms or the corresponding batch-norm scaling factors, while neglecting the sequential dependency between adjacent layers. In this paper, we further develop the norm-based importance estimation by taking the dependency between the adjacent layers into consideration. Besides, we propose a novel mechanism to dynamically control the sparsity-inducing regularization so as to achieve the desired sparsity. In this way, we can identify unimportant filters and search for the optimal…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neural Network Applications · Domain Adaptation and Few-Shot Learning · Human Pose and Action Recognition
MethodsPruning
