Equivariant Polynomials for Graph Neural Networks

TL;DR

This paper introduces a new hierarchy for measuring GNN expressive power based on equivariant polynomials, providing a full characterization, evaluation tools, and practical enhancements that improve performance on benchmarks.

Contribution

It offers a novel hierarchy based on equivariant polynomials, a complete characterization of equivariant graph polynomials, and methods to enhance GNN expressiveness with state-of-the-art results.

Findings

Full characterization of equivariant graph polynomials

Algorithmic tools for evaluating GNN expressiveness

Enhanced GNN architectures achieve state-of-the-art results

Abstract

Graph Neural Networks (GNN) are inherently limited in their expressive power. Recent seminal works (Xu et al., 2019; Morris et al., 2019b) introduced the Weisfeiler-Lehman (WL) hierarchy as a measure of expressive power. Although this hierarchy has propelled significant advances in GNN analysis and architecture developments, it suffers from several significant limitations. These include a complex definition that lacks direct guidance for model improvement and a WL hierarchy that is too coarse to study current GNNs. This paper introduces an alternative expressive power hierarchy based on the ability of GNNs to calculate equivariant polynomials of a certain degree. As a first step, we provide a full characterization of all equivariant graph polynomials by introducing a concrete basis, significantly generalizing previous results. Each basis element corresponds to a specific multi-graph, and its computation over some graph data input corresponds to a tensor contraction problem. Second, we propose algorithmic tools for evaluating the expressiveness of GNNs using tensor contraction sequences, and calculate the expressive power of popular GNNs. Finally, we enhance the expressivity of common GNN architectures by adding polynomial features or additional operations / aggregations inspired by our theory. These enhanced GNNs demonstrate state-of-the-art results in experiments across multiple graph learning benchmarks.