You Only Search Once: On Lightweight Differentiable Architecture Search for Resource-Constrained Embedded Platforms

TL;DR

LightNAS is a lightweight differentiable neural architecture search framework designed to efficiently find neural network architectures that meet specific performance constraints in resource-limited embedded systems, using only a single search run.

Contribution

We propose LightNAS, a novel one-time search method for hardware-aware neural architecture search that reduces the need for multiple trial-and-error searches, saving computational resources.

Findings

LightNAS outperforms previous NAS methods in resource-constrained scenarios.

It achieves the target performance constraints with a single search.

Experimental results demonstrate its efficiency and effectiveness.

Abstract

Benefiting from the search efficiency, differentiable neural architecture search (NAS) has evolved as the most dominant alternative to automatically design competitive deep neural networks (DNNs). We note that DNNs must be executed under strictly hard performance constraints in real-world scenarios, for example, the runtime latency on autonomous vehicles. However, to obtain the architecture that meets the given performance constraint, previous hardware-aware differentiable NAS methods have to repeat a plethora of search runs to manually tune the hyper-parameters by trial and error, and thus the total design cost increases proportionally. To resolve this, we introduce a lightweight hardware-aware differentiable NAS framework dubbed LightNAS, striving to find the required architecture that satisfies various performance constraints through a one-time search (i.e., \underline{\textit{you only search once}}). Extensive experiments are conducted to show the superiority of LightNAS over previous state-of-the-art methods.