DARTS+: Improved Differentiable Architecture Search with Early Stopping
Hanwen Liang, Shifeng Zhang, Jiacheng Sun, Xingqiu He, Weiran Huang,, Kechen Zhuang, Zhenguo Li

TL;DR
DARTS+ introduces an early stopping criterion to improve differentiable architecture search, preventing overfitting and collapse in search performance, leading to better results on benchmark datasets.
Contribution
The paper proposes a simple early stopping method for DARTS that effectively prevents performance collapse and overfitting during architecture search.
Findings
DARTS+ achieves 2.32% test error on CIFAR10.
DARTS+ achieves 14.87% test error on CIFAR100.
DARTS+ achieves 23.7% test error on ImageNet.
Abstract
Recently, there has been a growing interest in automating the process of neural architecture design, and the Differentiable Architecture Search (DARTS) method makes the process available within a few GPU days. However, the performance of DARTS is often observed to collapse when the number of search epochs becomes large. Meanwhile, lots of "{\em skip-connect}s" are found in the selected architectures. In this paper, we claim that the cause of the collapse is that there exists overfitting in the optimization of DARTS. Therefore, we propose a simple and effective algorithm, named "DARTS+", to avoid the collapse and improve the original DARTS, by "early stopping" the search procedure when meeting a certain criterion. We also conduct comprehensive experiments on benchmark datasets and different search spaces and show the effectiveness of our DARTS+ algorithm, and DARTS+ achieves …
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neural Network Applications · Domain Adaptation and Few-Shot Learning · Advanced Image and Video Retrieval Techniques
MethodsDifferentiable Architecture Search · Early Stopping
