Low-latency Mini-batch GNN Inference on CPU-FPGA Heterogeneous Platform

TL;DR

This paper presents a low-latency mini-batch GNN inference method on CPU-FPGA platforms using a novel adaptive hardware accelerator, achieving significant latency improvements over existing solutions.

Contribution

The paper introduces a flexible FPGA-based GNN accelerator with adaptive kernels and task scheduling, enabling efficient inference for various GNN models on heterogeneous platforms.

Findings

Achieves up to 50.8x latency reduction compared to CPU-only implementations.

Supports multiple GNN models including GCN, GraphSAGE, and GAT.

Demonstrates effective hiding of data communication overhead.

Abstract

Mini-batch inference of Graph Neural Networks (GNNs) is a key problem in many real-world applications. Recently, a GNN design principle of model depth-receptive field decoupling has been proposed to address the well-known issue of neighborhood explosion. Decoupled GNN models achieve higher accuracy than original models and demonstrate excellent scalability for mini-batch inference. We map Decoupled GNNs onto CPU-FPGA heterogeneous platforms to achieve low-latency mini-batch inference. On the FPGA platform, we design a novel GNN hardware accelerator with an adaptive datapath denoted Adaptive Computation Kernel (ACK) that can execute various computation kernels of GNNs with low-latency: (1) for dense computation kernels expressed as matrix multiplication, ACK works as a systolic array with fully localized connections, (2) for sparse computation kernels, ACK follows the scatter-gather paradigm and works as multiple parallel pipelines to support the irregular connectivity of graphs. The proposed task scheduling hides the CPU-FPGA data communication overhead to reduce the inference latency. We develop a fast design space exploration algorithm to generate a single accelerator for multiple target GNN models. We implement our accelerator on a state-of-the-art CPU-FPGA platform and evaluate the performance using three representative models (GCN, GraphSAGE, and GAT). Results show that our CPU-FPGA implementation achieves $21.4-50.8\times$, $2.9-21.6\times$, $4.7\times$ latency reduction compared with state-of-the-art implementations on CPU-only, CPU-GPU and CPU-FPGA platforms.