Sub-bit Neural Networks: Learning to Compress and Accelerate Binary Neural Networks

TL;DR

Sub-bit Neural Networks (SNNs) introduce a kernel-aware binary quantization method that significantly compresses and accelerates BNNs, achieving notable speed-ups and compression ratios on visual recognition tasks.

Contribution

The paper proposes a novel kernel-aware optimization framework for binary quantization, exploiting kernel subset distributions to improve BNN compression and acceleration.

Findings

SNNs achieve over 3x speed-up on ImageNet with ResNet models.

SNNs reduce model size by approximately 1.8 times compared to traditional BNNs.

Experimental validation on FPGA demonstrates practical deployment benefits.

Abstract

In the low-bit quantization field, training Binary Neural Networks (BNNs) is the extreme solution to ease the deployment of deep models on resource-constrained devices, having the lowest storage cost and significantly cheaper bit-wise operations compared to 32-bit floating-point counterparts. In this paper, we introduce Sub-bit Neural Networks (SNNs), a new type of binary quantization design tailored to compress and accelerate BNNs. SNNs are inspired by an empirical observation, showing that binary kernels learnt at convolutional layers of a BNN model are likely to be distributed over kernel subsets. As a result, unlike existing methods that binarize weights one by one, SNNs are trained with a kernel-aware optimization framework, which exploits binary quantization in the fine-grained convolutional kernel space. Specifically, our method includes a random sampling step generating layer-specific subsets of the kernel space, and a refinement step learning to adjust these subsets of binary kernels via optimization. Experiments on visual recognition benchmarks and the hardware deployment on FPGA validate the great potentials of SNNs. For instance, on ImageNet, SNNs of ResNet-18/ResNet-34 with 0.56-bit weights achieve 3.13/3.33 times runtime speed-up and 1.8 times compression over conventional BNNs with moderate drops in recognition accuracy. Promising results are also obtained when applying SNNs to binarize both weights and activations. Our code is available at https://github.com/yikaiw/SNN.