Technical report: Graph Neural Networks go Grammatical

TL;DR

This paper establishes a formal connection between algebraic languages and Graph Neural Networks using Context-Free Grammars, introduces a grammar reduction scheme, and presents a GNN model that is provably 3-WL compliant with demonstrated efficiency improvements.

Contribution

It introduces a novel framework linking CFGs to GNNs, including a grammar reduction method and a new GNN model, G$^2$N$^2$, that is provably 3-WL compliant.

Findings

G$^2$N$^2$ outperforms other 3-WL GNNs in efficiency.

Grammar reduction improves GNN performance.

The framework provides a formal basis for GNN design using CFGs.

Abstract

This paper introduces a framework for formally establishing a connection between a portion of an algebraic language and a Graph Neural Network (GNN). The framework leverages Context-Free Grammars (CFG) to organize algebraic operations into generative rules that can be translated into a GNN layer model. As CFGs derived directly from a language tend to contain redundancies in their rules and variables, we present a grammar reduction scheme. By applying this strategy, we define a CFG that conforms to the third-order Weisfeiler-Lehman (3-WL) test using MATLANG. From this 3-WL CFG, we derive a GNN model, named G$^2$N$^2$, which is provably 3-WL compliant. Through various experiments, we demonstrate the superior efficiency of G$^2$N$^2$ compared to other 3-WL GNNs across numerous downstream tasks. Specifically, one experiment highlights the benefits of grammar reduction within our framework.