On the choice of graph neural network architectures

TL;DR

This paper investigates how the performance of graph neural networks varies with data dimensionality and training data size, emphasizing the importance of diverse benchmarks for architecture choice.

Contribution

It provides empirical evidence that complex graph neural networks outperform simple models with more features and training data, and highlights the need for diverse benchmarks.

Findings

Complex GNNs outperform simple models with more data and features.

Simple models are near-optimal in high-dimensional, low-data settings.

Diverse benchmarks are crucial for evaluating GNN architectures.

Abstract

Seminal works on graph neural networks have primarily targeted semi-supervised node classification problems with few observed labels and high-dimensional signals. With the development of graph networks, this setup has become a de facto benchmark for a significant body of research. Interestingly, several works have recently shown that in this particular setting, graph neural networks do not perform much better than predefined low-pass filters followed by a linear classifier. However, when learning from little data in a high-dimensional space, it is not surprising that simple and heavily regularized methods are near-optimal. In this paper, we show empirically that in settings with fewer features and more training data, more complex graph networks significantly outperform simple models, and propose a few insights towards the proper choice of graph network architectures. We finally outline the importance of using sufficiently diverse benchmarks (including lower dimensional signals as well) when designing and studying new types of graph neural networks.