Greedy Network Enlarging
Chuanjian Liu, Kai Han, An Xiao, Yiping Deng, Wei Zhang, Chunjing Xu,, Yunhe Wang

TL;DR
This paper introduces a greedy method for enlarging CNNs by reallocating MACs across stages, leading to improved accuracy and state-of-the-art results on ImageNet.
Contribution
It proposes a stage-level capacity enlargement approach based on greedy reallocation of computations, improving upon uniform scaling methods.
Findings
Outperforms original EfficientNet scaling method.
Achieves 80.9% and 84.3% ImageNet top-1 accuracy on GhostNet at 600M and 4.4B MACs.
Demonstrates the effectiveness of stage-wise reallocation in CNN enlargement.
Abstract
Recent studies on deep convolutional neural networks present a simple paradigm of architecture design, i.e., models with more MACs typically achieve better accuracy, such as EfficientNet and RegNet. These works try to enlarge all the stages in the model with one unified rule by sampling and statistical methods. However, we observe that some network architectures have similar MACs and accuracies, but their allocations on computations for different stages are quite different. In this paper, we propose to enlarge the capacity of CNN models by improving their width, depth and resolution on stage level. Under the assumption that the top-performing smaller CNNs are a proper subcomponent of the top-performing larger CNNs, we propose an greedy network enlarging method based on the reallocation of computations. With step-by-step modifying the computations on different stages, the enlarged…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neural Network Applications · Domain Adaptation and Few-Shot Learning · Adversarial Robustness in Machine Learning
MethodsSigmoid Activation · Pointwise Convolution · Inverted Residual Block · Dense Connections · Convolution · Batch Normalization · Depthwise Convolution · Residual Connection · *Communicated@Fast*How Do I Communicate to Expedia? · Average Pooling
