Simple GNNs with Low Rank Non-parametric Aggregators

TL;DR

This paper demonstrates that simple GNN architectures with low-rank, non-parametric aggregators can perform competitively on semi-supervised node classification tasks, reducing complexity and hyperparameter tuning.

Contribution

It introduces a streamlined GNN design using non-parametric feature aggregation, challenging the need for complex, over-engineered models in common SSNC benchmarks.

Findings

Non-parametric regression methods perform well on sparse, directed networks

Simpler GNN architectures can match SOTA performance with less tuning

Evaluation changes may influence reported performance improvements

Abstract

We revisit recent spectral GNN approaches to semi-supervised node classification (SSNC). We posit that state-of-the-art (SOTA) GNN architectures may be over-engineered for common SSNC benchmark datasets (citation networks, page-page networks, etc.). By replacing feature aggregation with a non-parametric learner we are able to streamline the GNN design process and avoid many of the engineering complexities associated with SOTA hyperparameter selection (GNN depth, non-linearity choice, feature dropout probability, etc.). Our empirical experiments suggest conventional methods such as non-parametric regression are well suited for semi-supervised learning on sparse, directed networks and a variety of other graph types commonly found in SSNC benchmarks. Additionally, we bring attention to recent changes in evaluation conventions for SSNC benchmarking and how this may have partially contributed to rising performances over time.