HQNAS: Auto CNN deployment framework for joint quantization and architecture search

TL;DR

HQNAS is a novel framework that efficiently combines neural architecture search and quantization to optimize neural networks for edge devices, significantly reducing search time and latency while maintaining accuracy.

Contribution

The paper introduces HQNAS, a joint NAS and quantization framework using weight-sharing and bit-sharing, achieving faster search and better deployment performance on embedded systems.

Findings

Discoveries in 4 GPU hours on CIFAR10

Comparable models on ImageNet with 10% GPU time

Latency decreased by 1.8x with minimal accuracy loss

Abstract

Deep learning applications are being transferred from the cloud to edge with the rapid development of embedded computing systems. In order to achieve higher energy efficiency with the limited resource budget, neural networks(NNs) must be carefully designed in two steps, the architecture design and the quantization policy choice. Neural Architecture Search(NAS) and Quantization have been proposed separately when deploying NNs onto embedded devices. However, taking the two steps individually is time-consuming and leads to a sub-optimal final deployment. To this end, we propose a novel neural network design framework called Hardware-aware Quantized Neural Architecture Search(HQNAS) framework which combines the NAS and Quantization together in a very efficient manner using weight-sharing and bit-sharing. It takes only 4 GPU hours to discover an outstanding NN policy on CIFAR10. It also takes only %10 GPU time to generate a comparable model on Imagenet compared to the traditional NAS method with 1.8x decrease of latency and a negligible accuracy loss of only 0.7%. Besides, our method can be adapted in a lifelong situation where the neural network needs to evolve occasionally due to changes of local data, environment and user preference.