A Survey on Neural Architecture Search
Martin Wistuba, Ambrish Rawat, Tejaswini Pedapati

TL;DR
This survey comprehensively reviews neural architecture search methods, categorizing approaches, discussing search spaces and optimization algorithms, and exploring emerging directions like multi-objective search and automated data augmentation.
Contribution
It provides a unified formalism and detailed comparison of existing NAS methods, highlighting recent advances and future research directions.
Findings
Categorizes NAS methods based on search spaces and algorithms
Analyzes reinforcement learning, evolutionary, surrogate, and one-shot approaches
Identifies new trends like multi-objective search and automated data augmentation
Abstract
The growing interest in both the automation of machine learning and deep learning has inevitably led to the development of a wide variety of automated methods for neural architecture search. The choice of the network architecture has proven to be critical, and many advances in deep learning spring from its immediate improvements. However, deep learning techniques are computationally intensive and their application requires a high level of domain knowledge. Therefore, even partial automation of this process helps to make deep learning more accessible to both researchers and practitioners. With this survey, we provide a formalism which unifies and categorizes the landscape of existing methods along with a detailed analysis that compares and contrasts the different approaches. We achieve this via a comprehensive discussion of the commonly adopted architecture search spaces and architecture…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMachine Learning and Data Classification · Advanced Image and Video Retrieval Techniques · Reinforcement Learning in Robotics
