DARTS-: Robustly Stepping out of Performance Collapse Without Indicators
Xiangxiang Chu, Xiaoxing Wang, Bo Zhang, Shun Lu, Xiaolin Wei, Junchi, Yan

TL;DR
This paper introduces DARTS-, a method that enhances the robustness of differentiable architecture search by neutralizing the advantage of skip connections, thereby preventing performance collapse.
Contribution
DARTS- is a novel approach that mitigates performance collapse in DARTS by removing the bias of skip connections through an auxiliary connection, improving stability.
Findings
Significantly improves robustness across datasets
Reduces performance collapse incidents
Ensures fairer operation competition
Abstract
Despite the fast development of differentiable architecture search (DARTS), it suffers from long-standing performance instability, which extremely limits its application. Existing robustifying methods draw clues from the resulting deteriorated behavior instead of finding out its causing factor. Various indicators such as Hessian eigenvalues are proposed as a signal to stop searching before the performance collapses. However, these indicator-based methods tend to easily reject good architectures if the thresholds are inappropriately set, let alone the searching is intrinsically noisy. In this paper, we undertake a more subtle and direct approach to resolve the collapse. We first demonstrate that skip connections have a clear advantage over other candidate operations, where it can easily recover from a disadvantageous state and become dominant. We conjecture that this privilege is causing…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsSoftware Engineering Research · Software System Performance and Reliability · Anomaly Detection Techniques and Applications
