# Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hours

## TL;DR

Single-Path NAS introduces a fast, differentiable neural architecture search method that designs hardware-efficient ConvNets in under 4 hours, achieving state-of-the-art accuracy with minimal computational cost.

## Contribution

It proposes a novel single-path over-parameterized search space that drastically reduces search time and computational resources for designing mobile-efficient ConvNets.

## Key findings

- Achieves 74.96% top-1 accuracy on ImageNet with 79ms latency.
- Search process takes only 8 epochs (30 TPU-hours), up to 5,000x faster.
- Provides open-source code for reproducibility.

## Abstract

Can we automatically design a Convolutional Network (ConvNet) with the highest image classification accuracy under the runtime constraint of a mobile device? Neural architecture search (NAS) has revolutionized the design of hardware-efficient ConvNets by automating this process. However, the NAS problem remains challenging due to the combinatorially large design space, causing a significant searching time (at least 200 GPU-hours). To alleviate this complexity, we propose Single-Path NAS, a novel differentiable NAS method for designing hardware-efficient ConvNets in less than 4 hours. Our contributions are as follows: 1. Single-path search space: Compared to previous differentiable NAS methods, Single-Path NAS uses one single-path over-parameterized ConvNet to encode all architectural decisions with shared convolutional kernel parameters, hence drastically decreasing the number of trainable parameters and the search cost down to few epochs. 2. Hardware-efficient ImageNet classification: Single-Path NAS achieves 74.96% top-1 accuracy on ImageNet with 79ms latency on a Pixel 1 phone, which is state-of-the-art accuracy compared to NAS methods with similar constraints (<80ms). 3. NAS efficiency: Single-Path NAS search cost is only 8 epochs (30 TPU-hours), which is up to 5,000x faster compared to prior work. 4. Reproducibility: Unlike all recent mobile-efficient NAS methods which only release pretrained models, we open-source our entire codebase at: https://github.com/dstamoulis/single-path-nas.