Differential Evolution for Neural Architecture Search
Noor Awad, Neeratyoy Mallik, Frank Hutter

TL;DR
This paper introduces differential evolution as a new search strategy for neural architecture search, demonstrating its effectiveness and robustness across multiple NAS benchmarks using full evaluation methods.
Contribution
It applies differential evolution to NAS, providing a simple yet effective alternative to existing search strategies and showing improved results on standard benchmarks.
Findings
Differential evolution outperforms regularized evolution and Bayesian optimization.
It yields more robust results across 13 NAS benchmarks.
The approach is effective with full evaluation strategies.
Abstract
Neural architecture search (NAS) methods rely on a search strategy for deciding which architectures to evaluate next and a performance estimation strategy for assessing their performance (e.g., using full evaluations, multi-fidelity evaluations, or the one-shot model). In this paper, we focus on the search strategy. We introduce the simple yet powerful evolutionary algorithm of differential evolution to the NAS community. Using the simplest performance evaluation strategy of full evaluations, we comprehensively compare this search strategy to regularized evolution and Bayesian optimization and demonstrate that it yields improved and more robust results for 13 tabular NAS benchmarks based on NAS-Bench-101, NAS-Bench-1Shot1, NAS-Bench-201 and NAS-HPO bench.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMetaheuristic Optimization Algorithms Research · Neural Networks and Applications · Evolutionary Algorithms and Applications
