Generative Adversarial Neural Architecture Search
Seyed Saeed Changiz Rezaei, Fred X. Han, Di Niu, Mohammad Salameh,, Keith Mills, Shuo Lian, Wei Lu, and Shangling Jui

TL;DR
This paper introduces GA-NAS, a generative adversarial approach to neural architecture search that offers theoretical guarantees, improved stability, and the ability to explore large search spaces efficiently, outperforming existing methods.
Contribution
GA-NAS is a novel NAS method combining importance sampling and adversarial learning with reinforcement, providing convergence guarantees and enhanced reproducibility.
Findings
GA-NAS outperforms existing NAS benchmarks.
It effectively handles search constraints and spaces.
Can improve baseline models like EfficientNet.
Abstract
Despite the empirical success of neural architecture search (NAS) in deep learning applications, the optimality, reproducibility and cost of NAS schemes remain hard to assess. In this paper, we propose Generative Adversarial NAS (GA-NAS) with theoretically provable convergence guarantees, promoting stability and reproducibility in neural architecture search. Inspired by importance sampling, GA-NAS iteratively fits a generator to previously discovered top architectures, thus increasingly focusing on important parts of a large search space. Furthermore, we propose an efficient adversarial learning approach, where the generator is trained by reinforcement learning based on rewards provided by a discriminator, thus being able to explore the search space without evaluating a large number of architectures. Extensive experiments show that GA-NAS beats the best published results under several…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
MethodsPointwise Convolution · Depthwise Convolution · Depthwise Separable Convolution · Batch Normalization · Sigmoid Activation · REINFORCE · Dropout · *Communicated@Fast*How Do I Communicate to Expedia? · Average Pooling · Adam
