Binarizing by Classification: Is soft function really necessary?

TL;DR

This paper proposes a novel approach to binary neural network binarization by framing it as a binary classification problem using an MLP, eliminating the need for complex soft gradient functions and achieving state-of-the-art results in image classification and pose estimation.

Contribution

The authors introduce an MLP-based binarization method that adaptively learns gradients, outperforming traditional soft functions and simplifying the binarization process.

Findings

Achieved 65.7% top-1 accuracy on ImageNet with ResNet-34.

Binary pose estimation networks reached up to 60.6 mAP on COCO.

Demonstrated better performance-complexity trade-offs on real platforms.

Abstract

Binary neural networks leverage $\mathrm{Sign}$ function to binarize weights and activations, which require gradient estimators to overcome its non-differentiability and will inevitably bring gradient errors during backpropagation. Although many hand-designed soft functions have been proposed as gradient estimators to better approximate gradients, their mechanism is not clear and there are still huge performance gaps between binary models and their full-precision counterparts. To address these issues and reduce gradient error, we propose to tackle network binarization as a binary classification problem and use a multi-layer perceptron (MLP) as the classifier in the forward pass and gradient estimator in the backward pass. Benefiting from the MLP's theoretical capability to fit any continuous function, it can be adaptively learned to binarize networks and backpropagate gradients without any prior knowledge of soft functions. From this perspective, we further empirically justify that even a simple linear function can outperform previous complex soft functions. Extensive experiments demonstrate that the proposed method yields surprising performance both in image classification and human pose estimation tasks. Specifically, we achieve $65.7\%$ top-1 accuracy of ResNet-34 on ImageNet dataset, with an absolute improvement of $2.6\%$. Moreover, we take binarization as a lightweighting approach for pose estimation models and propose well-designed binary pose estimation networks SBPN and BHRNet. When evaluating on the challenging Microsoft COCO keypoint dataset, the proposed method enables binary networks to achieve a mAP of up to $60.6$ for the first time. Experiments conducted on real platforms demonstrate that BNN achieves a better balance between performance and computational complexity, especially when computational resources are extremely low.