Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks

TL;DR

This paper establishes the theoretical limitations of standard GNNs in distinguishing graph structures and introduces higher-order k-GNNs that incorporate complex graph features, improving classification and regression tasks.

Contribution

It relates GNNs to the Weisfeiler-Leman heuristic and proposes k-GNNs for capturing higher-order graph structures, enhancing expressiveness.

Findings

GNNs have the same expressiveness as 1-WL in distinguishing graphs.

Higher-order k-GNNs can utilize complex graph structures.

Experimental results show higher-order information improves graph classification.

Abstract

In recent years, graph neural networks (GNNs) have emerged as a powerful neural architecture to learn vector representations of nodes and graphs in a supervised, end-to-end fashion. Up to now, GNNs have only been evaluated empirically -- showing promising results. The following work investigates GNNs from a theoretical point of view and relates them to the $1$-dimensional Weisfeiler-Leman graph isomorphism heuristic ($1$-WL). We show that GNNs have the same expressiveness as the $1$-WL in terms of distinguishing non-isomorphic (sub-)graphs. Hence, both algorithms also have the same shortcomings. Based on this, we propose a generalization of GNNs, so-called $k$-dimensional GNNs ($k$-GNNs), which can take higher-order graph structures at multiple scales into account. These higher-order structures play an essential role in the characterization of social networks and molecule graphs. Our experimental evaluation confirms our theoretical findings as well as confirms that higher-order information is useful in the task of graph classification and regression.