SONAR: Joint Architecture and System Optimization Search
Elias J\"a\"asaari, Michelle Ma, Ameet Talwalkar, Tianqi Chen

TL;DR
SONAR is a joint search method that efficiently combines architecture and system optimization to find high-performance machine learning models tailored to specific hardware, significantly reducing search time.
Contribution
This paper introduces SONAR, a novel approach that interleaves architecture search and system optimization, enabling faster and more effective deployment of ML models on diverse hardware.
Findings
SONAR finds nearly optimal architectures 30 times faster than brute force.
Interleaving search processes improves efficiency and accuracy.
Early stopping enhances search speed without sacrificing quality.
Abstract
There is a growing need to deploy machine learning for different tasks on a wide array of new hardware platforms. Such deployment scenarios require tackling multiple challenges, including identifying a model architecture that can achieve a suitable predictive accuracy (architecture search), and finding an efficient implementation of the model to satisfy underlying hardware-specific systems constraints such as latency (system optimization search). Existing works treat architecture search and system optimization search as separate problems and solve them sequentially. In this paper, we instead propose to solve these problems jointly, and introduce a simple but effective baseline method called SONAR that interleaves these two search problems. SONAR aims to efficiently optimize for predictive accuracy and inference latency by applying early stopping to both search processes. Our experiments…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMachine Learning in Materials Science · Parallel Computing and Optimization Techniques · Machine Learning and Data Classification
MethodsEarly Stopping
