SCAN-Edge: Finding MobileNet-speed Hybrid Networks for Diverse Edge Devices via Hardware-Aware Evolutionary Search

TL;DR

SCAN-Edge introduces a hardware-aware evolutionary neural architecture search framework that designs hybrid networks optimized for diverse edge devices, effectively matching actual latency across CPUs, GPUs, and accelerators.

Contribution

It unifies NAS for various hardware by jointly searching for multiple operation types and employs a hardware-aware evolutionary algorithm to efficiently explore the large search space.

Findings

Hybrid networks match MobileNetV2 latency on multiple devices.

The framework effectively accommodates diverse hardware architectures.

Search process accelerates through improved search space quality.

Abstract

Designing low-latency and high-efficiency hybrid networks for a variety of low-cost commodity edge devices is both costly and tedious, leading to the adoption of hardware-aware neural architecture search (NAS) for finding optimal architectures. However, unifying NAS for a wide range of edge devices presents challenges due to the variety of hardware designs, supported operations, and compilation optimizations. Existing methods often fix the search space of architecture choices (e.g., activation, convolution, or self-attention) and estimate latency using hardware-agnostic proxies (e.g., FLOPs), which fail to achieve proclaimed latency across various edge devices. To address this issue, we propose SCAN-Edge, a unified NAS framework that jointly searches for self-attention, convolution, and activation to accommodate the wide variety of edge devices, including CPU-, GPU-, and hardware accelerator-based systems. To handle the large search space, SCAN-Edge relies on with a hardware-aware evolutionary algorithm that improves the quality of the search space to accelerate the sampling process. Experiments on large-scale datasets demonstrate that our hybrid networks match the actual MobileNetV2 latency for 224x224 input resolution on various commodity edge devices.