Towards Affordable, Adaptive and Automatic GNN Training on CPU-GPU Heterogeneous Platforms

TL;DR

This paper presents A3GNN, a framework that enables affordable, adaptive, and automatic GNN training on CPU-GPU heterogeneous platforms, significantly improving efficiency and resource utilization.

Contribution

It introduces a novel framework that combines locality-aware sampling, fine-grained scheduling, and reinforcement learning to optimize GNN training on heterogeneous platforms.

Findings

A3GNN achieves up to 1.8X throughput improvement over high-end GPUs.

It effectively balances throughput, memory, and accuracy through reinforcement learning.

The framework enables resource-constrained devices to perform competitively in GNN training.

Abstract

Graph Neural Networks (GNNs) have been widely adopted due to their strong performance. However, GNN training often relies on expensive, high-performance computing platforms, limiting accessibility for many tasks. Profiling of representative GNN workloads indicates that substantial efficiency gains are possible on resource-constrained devices by fully exploiting available resources. This paper introduces A3GNN, a framework for affordable, adaptive, and automatic GNN training on heterogeneous CPU-GPU platforms. It improves resource usage through locality-aware sampling and fine-grained parallelism scheduling. Moreover, it leverages reinforcement learning to explore the design space and achieve pareto-optimal trade-offs among throughput, memory footprint, and accuracy. Experiments show that A3GNN can bridge the performance gap, allowing seven Nvidia 2080Ti GPUs to outperform two A100 GPUs by up to 1.8X in throughput with minimal accuracy loss.