DGI: Easy and Efficient Inference for GNNs

TL;DR

DGI is a system that simplifies and accelerates GNN inference by automatically converting training code for layer-wise execution, significantly reducing inference time especially on large graphs.

Contribution

We introduce DGI, a system that automates the translation of GNN training code for efficient layer-wise inference, supporting out-of-core execution on large graphs.

Findings

DGI outperforms traditional layer-wise inference with over 1,000x speedup.

DGI is compatible with various GNN models and inference requests.

DGI supports out-of-core execution on graphs exceeding memory capacity.

Abstract

While many systems have been developed to train Graph Neural Networks (GNNs), efficient model inference and evaluation remain to be addressed. For instance, using the widely adopted node-wise approach, model evaluation can account for up to 94% of the time in the end-to-end training process due to neighbor explosion, which means that a node accesses its multi-hop neighbors. On the other hand, layer-wise inference avoids the neighbor explosion problem by conducting inference layer by layer such that the nodes only need their one-hop neighbors in each layer. However, implementing layer-wise inference requires substantial engineering efforts because users need to manually decompose a GNN model into layers for computation and split workload into batches to fit into device memory. In this paper, we develop Deep Graph Inference (DGI) -- a system for easy and efficient GNN model inference, which automatically translates the training code of a GNN model for layer-wise execution. DGI is general for various GNN models and different kinds of inference requests, and supports out-of-core execution on large graphs that cannot fit in CPU memory. Experimental results show that DGI consistently outperforms layer-wise inference across different datasets and hardware settings, and the speedup can be over 1,000x.