Optimizing Memory Efficiency of Graph Neural Networks on Edge Computing Platforms

TL;DR

This paper introduces a feature decomposition method to optimize memory usage and inference speed of GNNs on edge devices, effectively reducing memory consumption and preventing OOM errors.

Contribution

It presents a novel feature decomposition approach that significantly improves memory efficiency and inference speed of GNNs on edge platforms.

Findings

Speeds up GNN inference by up to 3x

Reduces peak memory usage by up to 5x

Mitigates Out-Of-Memory issues during inference

Abstract

Graph neural networks (GNN) have achieved state-of-the-art performance on various industrial tasks. However, the poor efficiency of GNN inference and frequent Out-Of-Memory (OOM) problem limit the successful application of GNN on edge computing platforms. To tackle these problems, a feature decomposition approach is proposed for memory efficiency optimization of GNN inference. The proposed approach could achieve outstanding optimization on various GNN models, covering a wide range of datasets, which speeds up the inference by up to 3x. Furthermore, the proposed feature decomposition could significantly reduce the peak memory usage (up to 5x in memory efficiency improvement) and mitigate OOM problems during GNN inference.