ReActXGB: A Hybrid Binary Convolutional Neural Network Architecture for Improved Performance and Computational Efficiency

TL;DR

ReActXGB introduces a hybrid model combining binary CNNs with XGBoost to enhance accuracy and efficiency, addressing the computational bottleneck of fully connected layers in BCNNs.

Contribution

The paper proposes replacing the fully convolutional layer of ReActNet-A with XGBoost, improving performance and reducing computational costs in BCNNs.

Findings

ReActXGB outperforms ReActNet-A by 1.47% in accuracy.

Achieves 7.14% reduction in FLOPs.

Reduces model size by 1.02%.

Abstract

Binary convolutional neural networks (BCNNs) provide a potential solution to reduce the memory requirements and computational costs associated with deep neural networks (DNNs). However, achieving a trade-off between performance and computational resources remains a significant challenge. Furthermore, the fully connected layer of BCNNs has evolved into a significant computational bottleneck. This is mainly due to the conventional practice of excluding the input layer and fully connected layer from binarization to prevent a substantial loss in accuracy. In this paper, we propose a hybrid model named ReActXGB, where we replace the fully convolutional layer of ReActNet-A with XGBoost. This modification targets to narrow the performance gap between BCNNs and real-valued networks while maintaining lower computational costs. Experimental results on the FashionMNIST benchmark demonstrate that ReActXGB outperforms ReActNet-A by 1.47% in top-1 accuracy, along with a reduction of 7.14% in floating-point operations (FLOPs) and 1.02% in model size.