TopoNAS: Boosting Search Efficiency of Gradient-based NAS via Topological Simplification
Danpei Zhao, Zhuoran Liu, Bo Yuan

TL;DR
TopoNAS introduces a topological simplification technique for gradient-based one-shot NAS, significantly reducing search time and memory while maintaining high accuracy across various architectures.
Contribution
It presents a model-agnostic, topological simplification approach that improves search efficiency in NAS by reducing redundancy and addressing non-linearity issues.
Findings
Reduces search time and memory usage in NAS.
Maintains high accuracy across different architectures.
Effective on NASBench201 benchmark.
Abstract
Improving search efficiency serves as one of the crucial objectives of Neural Architecture Search (NAS). However, many current approaches ignore the universality of the search strategy and fail to reduce the computational redundancy during the search process, especially in one-shot NAS architectures. Besides, current NAS methods show invalid reparameterization in non-linear search space, leading to poor efficiency in common search spaces like DARTS. In this paper, we propose TopoNAS, a model-agnostic approach for gradient-based one-shot NAS that significantly reduces searching time and memory usage by topological simplification of searchable paths. Firstly, we model the non-linearity in search spaces to reveal the parameterization difficulties. To improve the search efficiency, we present a topological simplification method and iteratively apply module-sharing strategies to simplify the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Image and Video Retrieval Techniques · Image Retrieval and Classification Techniques · Advanced Computing and Algorithms
MethodsDifferentiable Architecture Search
