Architectural Implications of Embedding Dimension during GCN on CPU and GPU

TL;DR

This paper investigates how embedding dimension choices in Graph Convolutional Networks affect performance on CPU and GPU architectures, considering factors like graph size and sampling.

Contribution

It provides a detailed analysis of GCN performance implications related to embedding dimension on CPU and GPU, guiding architectural decisions.

Findings

Embedding dimension significantly impacts GCN performance on both CPU and GPU.

Graph size and sampling strategies influence the efficiency of GCN inference.

Performance trade-offs vary between CPU and GPU implementations based on embedding choices.

Abstract

Graph Neural Networks (GNNs) are a class of neural networks designed to extract information from the graphical structure of data. Graph Convolutional Networks (GCNs) are a widely used type of GNN for transductive graph learning problems which apply convolution to learn information from graphs. GCN is a challenging algorithm from an architecture perspective due to inherent sparsity, low data reuse, and massive memory capacity requirements. Traditional neural algorithms exploit the high compute capacity of GPUs to achieve high performance for both inference and training. The architectural decision to use a GPU for GCN inference is a question explored in this work. GCN on both CPU and GPU was characterized in order to better understand the implications of graph size, embedding dimension, and sampling on performance.