Efficient Message Passing Architecture for GCN Training on HBM-based FPGAs with Orthogonal Topology On-Chip Networks

TL;DR

This paper introduces a novel message-passing architecture for GCN training on HBM-based FPGAs, utilizing a 4-D hypercube network and optimized backpropagation to improve performance and reduce memory bottlenecks.

Contribution

It presents a new FPGA-based GCN training architecture that enhances message passing efficiency and re-engineers backpropagation to lower memory and computational demands.

Findings

Achieved up to 1.81x performance improvement over state-of-the-art.

Utilized a 4-D hypercube network for efficient message routing.

Redesigned backpropagation to reduce memory and computational overhead.

Abstract

Graph Convolutional Networks (GCNs) are state-of-the-art deep learning models for representation learning on graphs. However, the efficient training of GCNs is hampered by constraints in memory capacity and bandwidth, compounded by the irregular data flow that results in communication bottlenecks. To address these challenges, we propose a message-passing architecture that leverages NUMA-based memory access properties and employs a parallel multicast routing algorithm based on a 4-D hypercube network within the accelerator for efficient message passing in graphs. Additionally, we have re-engineered the backpropagation algorithm specific to GCNs within our proposed accelerator. This redesign strategically mitigates the memory demands prevalent during the training phase and diminishes the computational overhead associated with the transposition of extensive matrices. Compared to the state-of-the-art HP-GNN architecture we achieved a performance improvement of $1.03\times \sim 1.81\times$.