Differentiable Architecture Search with Random Features
Xuanyang Zhang, Yonggang Li, Xiangyu Zhang, Yongtao Wang, Jian Sun

TL;DR
This paper addresses performance collapse in differentiable architecture search (DARTS) by introducing random features, which improve search stability and accuracy, leading to state-of-the-art results on CIFAR-10 and ImageNet datasets.
Contribution
The authors propose a novel approach using random features to mitigate DARTS performance collapse, with theoretical analysis and practical instantiations RF-DARTS and RF-PCDARTS.
Findings
RF-DARTS achieves 94.36% on CIFAR-10.
RF-DARTS attains 24.0% top-1 error on ImageNet.
RF-PCDARTS surpasses existing methods with 23.9% top-1 error on ImageNet.
Abstract
Differentiable architecture search (DARTS) has significantly promoted the development of NAS techniques because of its high search efficiency and effectiveness but suffers from performance collapse. In this paper, we make efforts to alleviate the performance collapse problem for DARTS from two aspects. First, we investigate the expressive power of the supernet in DARTS and then derive a new setup of DARTS paradigm with only training BatchNorm. Second, we theoretically find that random features dilute the auxiliary connection role of skip-connection in supernet optimization and enable search algorithm focus on fairer operation selection, thereby solving the performance collapse problem. We instantiate DARTS and PC-DARTS with random features to build an improved version for each named RF-DARTS and RF-PCDARTS respectively. Experimental results show that RF-DARTS obtains \textbf{94.36\%}…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMachine Learning and Data Classification · Advanced Neural Network Applications · Anomaly Detection Techniques and Applications
MethodsTest · Differentiable Architecture Search
