Efficient Architecture Search for Diverse Tasks
Junhong Shen, Mikhail Khodak, Ameet Talwalkar

TL;DR
This paper introduces DASH, a fast differentiable NAS method that searches for optimal kernel sizes and dilations in CNNs, enabling efficient and effective AutoML across diverse tasks beyond computer vision.
Contribution
The paper proposes DASH, a novel differentiable NAS algorithm using Fourier diagonalization, achieving faster search times and broad applicability for diverse problem domains.
Findings
DASH outperforms state-of-the-art AutoML methods on multiple tasks.
DASH achieves near-training speed with high accuracy on several applications.
The approach generalizes well across domains like PDE solving and protein folding.
Abstract
While neural architecture search (NAS) has enabled automated machine learning (AutoML) for well-researched areas, its application to tasks beyond computer vision is still under-explored. As less-studied domains are precisely those where we expect AutoML to have the greatest impact, in this work we study NAS for efficiently solving diverse problems. Seeking an approach that is fast, simple, and broadly applicable, we fix a standard convolutional network (CNN) topology and propose to search for the right kernel sizes and dilations its operations should take on. This dramatically expands the model's capacity to extract features at multiple resolutions for different types of data while only requiring search over the operation space. To overcome the efficiency challenges of naive weight-sharing in this search space, we introduce DASH, a differentiable NAS algorithm that computes the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsMachine Learning and Data Classification · Machine Learning in Bioinformatics · Machine Learning in Materials Science
MethodsDifferentiable Neural Architecture Search
