Searching Similarity Measure for Binarized Neural Networks

TL;DR

This paper introduces an automatic genetic algorithm-based method to optimize the similarity measure in Binarized Neural Networks, significantly improving their accuracy on benchmark datasets.

Contribution

It proposes a novel genetic algorithm approach to search for BNN-specific similarity measures, addressing a key component previously designed for DNNs.

Findings

Achieved up to 3.39% accuracy improvement on CIFAR datasets

Most identified similarity measures outperform the standard cross-correlation

Demonstrated effectiveness across multiple network architectures

Abstract

Being a promising model to be deployed in resource-limited devices, Binarized Neural Networks (BNNs) have drawn extensive attention from both academic and industry. However, comparing to the full-precision deep neural networks (DNNs), BNNs suffer from non-trivial accuracy degradation, limiting its applicability in various domains. This is partially because existing network components, such as the similarity measure, are specially designed for DNNs, and might be sub-optimal for BNNs. In this work, we focus on the key component of BNNs -- the similarity measure, which quantifies the distance between input feature maps and filters, and propose an automatic searching method, based on genetic algorithm, for BNN-tailored similarity measure. Evaluation results on Cifar10 and Cifar100 using ResNet, NIN and VGG show that most of the identified similarty measure can achieve considerable accuracy improvement (up to 3.39%) over the commonly-used cross-correlation approach.