IS-DARTS: Stabilizing DARTS through Precise Measurement on Candidate Importance
Hongyi He, Longjun Liu, Haonan Zhang, Nanning Zheng

TL;DR
IS-DARTS enhances DARTS by addressing biased importance estimation and optimizing the search process, leading to more robust neural architectures with improved performance, as validated on benchmark datasets.
Contribution
The paper introduces IS-DARTS, a novel method that stabilizes DARTS by using information-based measurements and theoretical analysis to improve operation selection and supernet optimization.
Findings
IS-DARTS outperforms baseline DARTS on NAS-Bench-201.
The method reduces architecture performance collapse.
Theoretical analysis supports the necessity of supernet width shrinking.
Abstract
Among existing Neural Architecture Search methods, DARTS is known for its efficiency and simplicity. This approach applies continuous relaxation of network representation to construct a weight-sharing supernet and enables the identification of excellent subnets in just a few GPU days. However, performance collapse in DARTS results in deteriorating architectures filled with parameter-free operations and remains a great challenge to the robustness. To resolve this problem, we reveal that the fundamental reason is the biased estimation of the candidate importance in the search space through theoretical and experimental analysis, and more precisely select operations via information-based measurements. Furthermore, we demonstrate that the excessive concern over the supernet and inefficient utilization of data in bi-level optimization also account for suboptimal results. We adopt a more…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neural Network Applications · Machine Learning and Algorithms · Neural Networks and Applications
MethodsDifferentiable Architecture Search
