Extensible Proxy for Efficient NAS
Yuhong Li, Jiajie Li, Cong Han, Pan Li, Jinjun Xiong, Deming Chen

TL;DR
This paper introduces Eproxy, an extensible, near-zero-cost neural architecture performance predictor using self-supervised, few-shot training, adaptable to various search spaces and downstream tasks, significantly reducing NAS computational costs.
Contribution
The paper proposes Eproxy, a novel extensible proxy for NAS that employs a barrier layer and discrete proxy search to adapt to different tasks and search spaces with minimal training.
Findings
Eproxy achieves near-zero-cost performance prediction.
Eproxy is adaptable to multiple search spaces and downstream tasks.
Experimental results confirm the effectiveness of Eproxy and DPS.
Abstract
Neural Architecture Search (NAS) has become a de facto approach in the recent trend of AutoML to design deep neural networks (DNNs). Efficient or near-zero-cost NAS proxies are further proposed to address the demanding computational issues of NAS, where each candidate architecture network only requires one iteration of backpropagation. The values obtained from the proxies are considered the predictions of architecture performance on downstream tasks. However, two significant drawbacks hinder the extended usage of Efficient NAS proxies. (1) Efficient proxies are not adaptive to various search spaces. (2) Efficient proxies are not extensible to multi-modality downstream tasks. Based on the observations, we design a Extensible proxy (Eproxy) that utilizes self-supervised, few-shot training (i.e., 10 iterations of backpropagation) which yields near-zero costs. The key component that makes…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMachine Learning and Data Classification · Machine Learning and Algorithms · Adversarial Robustness in Machine Learning
MethodsConvolution
