A Computing Kernel for Network Binarization on PyTorch

TL;DR

This paper introduces a new computing kernel for network binarization in PyTorch, significantly accelerating inference on GPU and CPU, and facilitating deployment of efficient neural networks on low-power devices.

Contribution

It develops the first PyTorch computing kernel supporting 1-bit xnor and bitcount operations for network binarization.

Findings

3x inference acceleration on GPU

4.5x inference acceleration on CPU

Enables efficient deployment of binarized neural networks

Abstract

Deep Neural Networks have now achieved state-of-the-art results in a wide range of tasks including image classification, object detection and so on. However, they are both computation consuming and memory intensive, making them difficult to deploy on low-power devices. Network binarization is one of the existing effective techniques for model compression and acceleration, but there is no computing kernel yet to support it on PyTorch. In this paper we developed a computing kernel supporting 1-bit xnor and bitcount computation on PyTorch. Experimental results show that our kernel could accelerate the inference of the binarized neural network by 3 times in GPU and by 4.5 times in CPU compared with the control group.