AutoSlim: Towards One-Shot Architecture Search for Channel Numbers
Jiahui Yu, Thomas Huang

TL;DR
AutoSlim introduces a one-shot method to optimize neural network channel configurations by training a single slimmable network, leading to improved accuracy and resource efficiency across various models and constraints.
Contribution
The paper proposes AutoSlim, a novel one-shot approach for channel number optimization that outperforms existing methods in accuracy and resource utilization without extensive search.
Findings
AutoSlim achieves higher accuracy than default configurations.
It outperforms recent channel pruning and neural architecture search methods.
Significant accuracy improvements on ImageNet across multiple models.
Abstract
We study how to set channel numbers in a neural network to achieve better accuracy under constrained resources (e.g., FLOPs, latency, memory footprint or model size). A simple and one-shot solution, named AutoSlim, is presented. Instead of training many network samples and searching with reinforcement learning, we train a single slimmable network to approximate the network accuracy of different channel configurations. We then iteratively evaluate the trained slimmable model and greedily slim the layer with minimal accuracy drop. By this single pass, we can obtain the optimized channel configurations under different resource constraints. We present experiments with MobileNet v1, MobileNet v2, ResNet-50 and RL-searched MNasNet on ImageNet classification. We show significant improvements over their default channel configurations. We also achieve better accuracy than recent channel pruning…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neural Network Applications · Adversarial Robustness in Machine Learning · Domain Adaptation and Few-Shot Learning
MethodsPruning · Residual Connection · MobileNetV1 · Tether Customer Service Number +1-833-534-1729 · Max Pooling · Bottleneck Residual Block · Residual Block · Kaiming Initialization · Bitcoin Customer Service Number +1-833-534-1729 · Tanh Activation
