Exploiting On-chip Heterogeneity of Versal Architecture for GNN Inference Acceleration

TL;DR

This paper presents a novel approach to accelerate GNN inference by leveraging the heterogeneous capabilities of AMD Versal ACAP architecture, combining custom hardware modules and dynamic task mapping to exploit data sparsity.

Contribution

It introduces a runtime kernel mapping strategy and custom hardware modules that utilize the heterogeneous architecture for efficient GNN inference acceleration.

Findings

Achieves up to 162.42x speedup over state-of-the-art methods.

Demonstrates significant performance improvements on various models and datasets.

Provides a flexible approach for dynamic sparsity exploitation in GNN inference.

Abstract

Graph Neural Networks (GNNs) have revolutionized many Machine Learning (ML) applications, such as social network analysis, bioinformatics, etc. GNN inference can be accelerated by exploiting data sparsity in the input graph, vertex features, and intermediate data in GNN computations. For dynamic sparsity exploitation, we leverage the heterogeneous computing capabilities of AMD Versal ACAP architecture to accelerate GNN inference. We develop a custom hardware module that executes the sparse primitives of the computation kernel on the Programmable Logic (PL) and efficiently computes the dense primitives using the AI Engine (AIE). To exploit data sparsity during inference, we devise a runtime kernel mapping strategy that dynamically assigns computation tasks to the PL and AIE based on data sparsity. Our implementation on the VCK5000 ACAP platform leads to superior performance compared with the state-of-the-art implementations on CPU, GPU, ACAP, and other custom GNN accelerators. Compared with these implementations, we achieve significant average runtime speedup across various models and datasets of 162.42x, 17.01x, 9.90x, and 27.23x, respectively. Furthermore, for Graph Convolutional Network (GCN) inference, our approach leads to a speedup of 3.9-96.7x compared to designs using PL only on the same ACAP device.