A First Step Towards Runtime Analysis of Evolutionary Neural Architecture Search
Zeqiong Lv, Chao Qian, Yanan Sun

TL;DR
This paper initiates the theoretical analysis of evolutionary neural architecture search by defining a simple classification problem and analyzing the runtime of mutation-based algorithms, providing foundational insights into their efficiency.
Contribution
It introduces a formal framework for analyzing ENAS, including a specific problem setup and runtime bounds for mutation algorithms, bridging empirical success with theoretical understanding.
Findings
Both local and global mutations find the optimum in expected linear time
Local and global mutations perform similarly on the defined problem
Empirical results confirm the theoretical equivalence of mutation operators
Abstract
Evolutionary neural architecture search (ENAS) employs evolutionary algorithms to find high-performing neural architectures automatically, and has achieved great success. However, compared to the empirical success, its rigorous theoretical analysis has yet to be touched. This work goes preliminary steps toward the mathematical runtime analysis of ENAS. In particular, we define a binary classification problem , and formulate an explicit fitness function to represent the relationship between neural architecture and classification accuracy. Furthermore, we consider (1+1)-ENAS algorithm with mutation to optimize the neural architecture, and obtain the following runtime bounds: both the local and global mutations find the optimum in an expected runtime of , where is the problem size. The theoretical results show that the local and global mutations achieve…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNeural Networks and Applications · Evolutionary Algorithms and Applications · Robotic Path Planning Algorithms
