Characterizing the Efficiency of Graph Neural Network Frameworks with a Magnifying Glass

TL;DR

This paper provides a comprehensive analysis of the performance and energy efficiency of two mainstream GNN frameworks across various models and benchmarks, highlighting their strengths and areas for improvement.

Contribution

It offers an in-depth benchmarking study of GNN frameworks, focusing on runtime, power consumption, and the impact of different sampling techniques.

Findings

Framework performance varies significantly with sampling techniques.

Power consumption is influenced by implementation details.

Benchmark results guide future GNN framework optimization.

Abstract

Graph neural networks (GNNs) have received great attention due to their success in various graph-related learning tasks. Several GNN frameworks have then been developed for fast and easy implementation of GNN models. Despite their popularity, they are not well documented, and their implementations and system performance have not been well understood. In particular, unlike the traditional GNNs that are trained based on the entire graph in a full-batch manner, recent GNNs have been developed with different graph sampling techniques for mini-batch training of GNNs on large graphs. While they improve the scalability, their training times still depend on the implementations in the frameworks as sampling and its associated operations can introduce non-negligible overhead and computational cost. In addition, it is unknown how much the frameworks are 'eco-friendly' from a green computing perspective. In this paper, we provide an in-depth study of two mainstream GNN frameworks along with three state-of-the-art GNNs to analyze their performance in terms of runtime and power/energy consumption. We conduct extensive benchmark experiments at several different levels and present detailed analysis results and observations, which could be helpful for further improvement and optimization.