Probabilistic Neural Architecture Search
Francesco Paolo Casale, Jonathan Gordon, Nicolo Fusi

TL;DR
This paper introduces PARSEC, a probabilistic neural architecture search method that significantly reduces memory usage and computational cost, enabling efficient search over complex architectures and larger datasets.
Contribution
PARSEC presents a memory-efficient probabilistic framework for neural architecture search that transfers learned distributions across different dataset sizes.
Findings
Outperforms methods with double the computational cost on CIFAR-10.
Matches performance of methods with 1000x higher computational cost on ImageNet.
Requires only as much memory as training a single architecture.
Abstract
In neural architecture search (NAS), the space of neural network architectures is automatically explored to maximize predictive accuracy for a given task. Despite the success of recent approaches, most existing methods cannot be directly applied to large scale problems because of their prohibitive computational complexity or high memory usage. In this work, we propose a Probabilistic approach to neural ARchitecture SEarCh (PARSEC) that drastically reduces memory requirements while maintaining state-of-the-art computational complexity, making it possible to directly search over more complex architectures and larger datasets. Our approach only requires as much memory as is needed to train a single architecture from our search space. This is due to a memory-efficient sampling procedure wherein we learn a probability distribution over high-performing neural network architectures.…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neural Network Applications · Adversarial Robustness in Machine Learning · Machine Learning and Data Classification
MethodsSigmoid Activation · Tanh Activation · Softmax · Long Short-Term Memory
