NAS-BNN: Neural Architecture Search for Binary Neural Networks

TL;DR

This paper introduces NAS-BNN, a neural architecture search method tailored for binary neural networks, significantly improving their accuracy and efficiency on tasks like image classification and object detection.

Contribution

The paper proposes a novel NAS scheme specifically designed for BNNs, including a tailored search space and training strategies, leading to state-of-the-art performance.

Findings

Achieved 68.20% top-1 accuracy on ImageNet with 57M OPs.

Outperformed previous BNNs across various operations (20M to 200M OPs).

Set new state-of-the-art results in object detection with 31.6% mAP on MS COCO.

Abstract

Binary Neural Networks (BNNs) have gained extensive attention for their superior inferencing efficiency and compression ratio compared to traditional full-precision networks. However, due to the unique characteristics of BNNs, designing a powerful binary architecture is challenging and often requires significant manpower. A promising solution is to utilize Neural Architecture Search (NAS) to assist in designing BNNs, but current NAS methods for BNNs are relatively straightforward and leave a performance gap between the searched models and manually designed ones. To address this gap, we propose a novel neural architecture search scheme for binary neural networks, named NAS-BNN. We first carefully design a search space based on the unique characteristics of BNNs. Then, we present three training strategies, which significantly enhance the training of supernet and boost the performance of all subnets. Our discovered binary model family outperforms previous BNNs for a wide range of operations (OPs) from 20M to 200M. For instance, we achieve 68.20% top-1 accuracy on ImageNet with only 57M OPs. In addition, we validate the transferability of these searched BNNs on the object detection task, and our binary detectors with the searched BNNs achieve a novel state-of-the-art result, e.g., 31.6% mAP with 370M OPs, on MS COCO dataset. The source code and models will be released at https://github.com/VDIGPKU/NAS-BNN.