Bi-GCN: Binary Graph Convolutional Network

TL;DR

Bi-GCN introduces a binarized graph neural network that significantly reduces memory usage and accelerates inference while maintaining comparable performance to full-precision models, enabling efficient processing of large attributed graphs.

Contribution

This paper proposes the first binarized GNN, Bi-GCN, which reduces memory and computation costs by binarizing parameters and features, and introduces a new training method for effective learning.

Findings

Memory consumption reduced by ~30x.

Inference speed increased by ~47x.

Performance comparable to full-precision GNNs.

Abstract

Graph Neural Networks (GNNs) have achieved tremendous success in graph representation learning. Unfortunately, current GNNs usually rely on loading the entire attributed graph into network for processing. This implicit assumption may not be satisfied with limited memory resources, especially when the attributed graph is large. In this paper, we pioneer to propose a Binary Graph Convolutional Network (Bi-GCN), which binarizes both the network parameters and input node features. Besides, the original matrix multiplications are revised to binary operations for accelerations. According to the theoretical analysis, our Bi-GCN can reduce the memory consumption by an average of ~30x for both the network parameters and input data, and accelerate the inference speed by an average of ~47x, on the citation networks. Meanwhile, we also design a new gradient approximation based back-propagation method to train our Bi-GCN well. Extensive experiments have demonstrated that our Bi-GCN can give a comparable performance compared to the full-precision baselines. Besides, our binarization approach can be easily applied to other GNNs, which has been verified in the experiments.