Model-Architecture Co-Design for High Performance Temporal GNN Inference on FPGA

TL;DR

This paper introduces a co-designed FPGA hardware and simplified model architecture for high-performance inference of temporal graph neural networks, achieving efficiency through optimized attention computation, neighbor pruning, and hardware techniques.

Contribution

It presents a novel combined model-architecture design for efficient TGNN inference on FPGAs, including lightweight attention, neighbor pruning, and hardware optimizations.

Findings

Achieved high throughput on real-world datasets.

Reduced computation and memory access through pruning and hardware design.

Maintained accuracy with knowledge distillation.

Abstract

Temporal Graph Neural Networks (TGNNs) are powerful models to capture temporal, structural, and contextual information on temporal graphs. The generated temporal node embeddings outperform other methods in many downstream tasks. Real-world applications require high performance inference on real-time streaming dynamic graphs. However, these models usually rely on complex attention mechanisms to capture relationships between temporal neighbors. In addition, maintaining vertex memory suffers from intrinsic temporal data dependency that hinders task-level parallelism, making it inefficient on general-purpose processors. In this work, we present a novel model-architecture co-design for inference in memory-based TGNNs on FPGAs. The key modeling optimizations we propose include a light-weight method to compute attention scores and a related temporal neighbor pruning strategy to further reduce computation and memory accesses. These are holistically coupled with key hardware optimizations that leverage FPGA hardware. We replace the temporal sampler with an on-chip FIFO based hardware sampler and the time encoder with a look-up-table. We train our simplified models using knowledge distillation to ensure similar accuracy vis-\'a-vis the original model. Taking advantage of the model optimizations, we propose a principled hardware architecture using batching, pipelining, and prefetching techniques to further improve the performance. We also propose a hardware mechanism to ensure the chronological vertex updating without sacrificing the computation parallelism. We evaluate the performance of the proposed hardware accelerator on three real-world datasets.