Automated Dominative Subspace Mining for Efficient Neural Architecture Search

TL;DR

This paper introduces DSM-NAS, a neural architecture search method that efficiently finds promising architectures by mining and searching within automatically identified subspaces, significantly reducing search time and improving results.

Contribution

The paper proposes a novel subspace mining approach for NAS that enhances search efficiency and effectiveness by focusing on promising subspaces and initializing searches with well-designed architectures.

Findings

Reduces search cost compared to state-of-the-art methods

Discovers better architectures in benchmark spaces

Effective subspace mining improves search performance

Abstract

Neural Architecture Search (NAS) aims to automatically find effective architectures within a predefined search space. However, the search space is often extremely large. As a result, directly searching in such a large search space is non-trivial and also very time-consuming. To address the above issues, in each search step, we seek to limit the search space to a small but effective subspace to boost both the search performance and search efficiency. To this end, we propose a novel Neural Architecture Search method via Dominative Subspace Mining (DSM-NAS) that finds promising architectures in automatically mined subspaces. Specifically, we first perform a global search, i.e ., dominative subspace mining, to find a good subspace from a set of candidates. Then, we perform a local search within the mined subspace to find effective architectures. More critically, we further boost search performance by taking well-designed/ searched architectures to initialize candidate subspaces. Experimental results demonstrate that DSM-NAS not only reduces the search cost but also discovers better architectures than state-of-the-art methods in various benchmark search spaces.