A Data-Driven Approach to Dataflow-Aware Online Scheduling for Graph Neural Network Inference

TL;DR

This paper presents a data-driven framework that predicts optimal dataflows for GNN inference, enabling an online scheduler to significantly improve performance across diverse graph types.

Contribution

It introduces a novel latency prediction model for GNN dataflows and an online scheduling algorithm that adapts to graph diversity, enhancing accelerator efficiency.

Findings

Predicts dataflow latency with up to 91.28% accuracy

Achieves up to 3.17x speedup in mean completion time

Reduces execution time variability across graph types

Abstract

Graph Neural Networks (GNNs) have shown significant promise in various domains, such as recommendation systems, bioinformatics, and network analysis. However, the irregularity of graph data poses unique challenges for efficient computation, leading to the development of specialized GNN accelerator architectures that surpass traditional CPU and GPU performance. Despite this, the structural diversity of input graphs results in varying performance across different GNN accelerators, depending on their dataflows. This variability in performance due to differing dataflows and graph properties remains largely unexplored, limiting the adaptability of GNN accelerators. To address this, we propose a data-driven framework for dataflow-aware latency prediction in GNN inference. Our approach involves training regressors to predict the latency of executing specific graphs on particular dataflows, using simulations on synthetic graphs. Experimental results indicate that our regressors can predict the optimal dataflow for a given graph with up to 91.28% accuracy and a Mean Absolute Percentage Error (MAPE) of 3.78%. Additionally, we introduce an online scheduling algorithm that uses these regressors to enhance scheduling decisions. Our experiments demonstrate that this algorithm achieves up to $3.17\times$ speedup in mean completion time and $6.26\times$ speedup in mean execution time compared to the best feasible baseline across all datasets.