Neural Inheritance Relation Guided One-Shot Layer Assignment Search
Rang Meng, Weijie Chen, Di Xie, Yuan Zhang, Shiliang Pu

TL;DR
This paper introduces a novel neural inheritance relation in layer assignment, enabling an efficient one-shot search method that significantly improves neural architecture search performance across multiple datasets.
Contribution
It uncovers a neural inheritance relation in layer assignment and proposes an inheritance-guided one-shot search approach that reduces search space and enhances efficiency.
Findings
The inheritance relation holds across different network depths.
The proposed method achieves superior results compared to handcrafted architectures.
Experiments on CIFAR-100, Tiny-ImageNet, and ImageNet validate effectiveness.
Abstract
Layer assignment is seldom picked out as an independent research topic in neural architecture search. In this paper, for the first time, we systematically investigate the impact of different layer assignments to the network performance by building an architecture dataset of layer assignment on CIFAR-100. Through analyzing this dataset, we discover a neural inheritance relation among the networks with different layer assignments, that is, the optimal layer assignments for deeper networks always inherit from those for shallow networks. Inspired by this neural inheritance relation, we propose an efficient one-shot layer assignment search approach via inherited sampling. Specifically, the optimal layer assignment searched in the shallow network can be provided as a strong sampling priori to train and search the deeper ones in supernet, which extremely reduces the network search space.…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neural Network Applications · Domain Adaptation and Few-Shot Learning · Advanced Image and Video Retrieval Techniques
