BOMP-NAS: Bayesian Optimization Mixed Precision NAS

TL;DR

BOMP-NAS introduces a Bayesian optimization-based mixed-precision neural architecture search method that efficiently finds compact, high-performance neural networks with significantly reduced search time and model size.

Contribution

It combines Bayesian optimization with mixed-precision quantization and quantization-aware fine-tuning to improve neural architecture search efficiency and performance.

Findings

Achieves nearly 50% model size reduction on CIFAR-10.

Reduces search time by 6x compared to related methods.

Finds state-of-the-art networks with lower design costs.

Abstract

Bayesian Optimization Mixed-Precision Neural Architecture Search (BOMP-NAS) is an approach to quantization-aware neural architecture search (QA-NAS) that leverages both Bayesian optimization (BO) and mixed-precision quantization (MP) to efficiently search for compact, high performance deep neural networks. The results show that integrating quantization-aware fine-tuning (QAFT) into the NAS loop is a necessary step to find networks that perform well under low-precision quantization: integrating it allows a model size reduction of nearly 50\% on the CIFAR-10 dataset. BOMP-NAS is able to find neural networks that achieve state of the art performance at much lower design costs. This study shows that BOMP-NAS can find these neural networks at a 6x shorter search time compared to the closest related work.