Directed-Weighting Group Lasso for Eltwise Blocked CNN Pruning
Ke Zhan, Shimiao Jiang, Yu Bai, Yi Li, Xu Liu, Zhuoran Xu

TL;DR
This paper introduces Directed-Weighting Group Lasso (DWGL), a novel pruning method for eltwise blocked CNNs that enhances filter pruning efficiency and network compression while maintaining accuracy.
Contribution
The work proposes DWGL, which enforces index-wise incremental coefficients to improve filter pruning in eltwise layers, boosting compression rates in CNNs.
Findings
Achieved 75.34% compression on ResNet-56 with minimal accuracy loss.
Attained 53% compression on ResNet-50 on ImageNet, doubling inference speed.
Demonstrated effective pruning across multiple ResNet architectures.
Abstract
Eltwise layer is a commonly used structure in the multi-branch deep learning network. In a filter-wise pruning procedure, due to the specific operation of the eltwise layer, all its previous convolutional layers should vote for which filters by index should be pruned. Since only an intersection of the voted filters is pruned, the compression rate is limited. This work proposes a method called Directed-Weighting Group Lasso (DWGL), which enforces an index-wise incremental (directed) coefficient on the filterlevel group lasso items, so that the low index filters getting high activation tend to be kept while the high index ones tend to be pruned. When using DWGL, much fewer filters are retained during the voting process and the compression rate can be boosted. The paper test the proposed method on the ResNet series networks. On CIFAR-10, it achieved a 75.34% compression rate on ResNet-56…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neural Network Applications · Adversarial Robustness in Machine Learning · Anomaly Detection Techniques and Applications
MethodsPruning · Test · Average Pooling · *Communicated@Fast*How Do I Communicate to Expedia? · 1x1 Convolution · Batch Normalization · Bottleneck Residual Block · Global Average Pooling · Residual Block · Kaiming Initialization
