On the Expressive Power of Graph Neural Networks

TL;DR

This paper reviews the theoretical aspects of Graph Neural Networks' expressive power, comparing different definitions and their implications for GNN architecture design, with a focus on their ability to distinguish graph structures and approximate functions.

Contribution

It provides a comprehensive overview of the various notions of GNN expressiveness and insights into how these influence architecture development.

Findings

Different definitions of GNN expressiveness are complementary.

GNNs' ability to distinguish graph structures varies with architecture.

Theoretical insights guide better GNN design choices.

Abstract

The study of Graph Neural Networks has received considerable interest in the past few years. By extending deep learning to graph-structured data, GNNs can solve a diverse set of tasks in fields including social science, chemistry, and medicine. The development of GNN architectures has largely been focused on improving empirical performance on tasks like node or graph classification. However, a line of recent work has instead sought to find GNN architectures that have desirable theoretical properties - by studying their expressive power and designing architectures that maximize this expressiveness. While there is no consensus on the best way to define the expressiveness of a GNN, it can be viewed from several well-motivated perspectives. Perhaps the most natural approach is to study the universal approximation properties of GNNs, much in the way that this has been studied extensively for MLPs. Another direction focuses on the extent to which GNNs can distinguish between different graph structures, relating this to the graph isomorphism test. Besides, a GNN's ability to compute graph properties such as graph moments has been suggested as another form of expressiveness. All of these different definitions are complementary and have yielded different recommendations for GNN architecture choices. In this paper, we would like to give an overview of the notion of "expressive power" of GNNs and provide some valuable insights regarding the design choices of GNNs.