On the under-reaching phenomenon in message-passing neural PDE solvers: revisiting the CFL condition

TL;DR

This paper establishes precise lower bounds on message passing iterations in GNNs for PDE solving, linking physical problem characteristics to GNN requirements, and demonstrating the bounds' effectiveness across various PDE types.

Contribution

It introduces sharp lower bounds for message passing in GNNs solving PDEs, reducing hyperparameter tuning and improving understanding of information propagation limits.

Findings

Lower bounds depend on physical constants and discretization

Insufficient message passing leads to poor solutions

Bounds are validated across multiple PDE examples

Abstract

This paper proposes sharp lower bounds for the number of message passing iterations required in graph neural networks (GNNs) when solving partial differential equations (PDE). This significantly reduces the need for exhaustive hyperparameter tuning. Bounds are derived for the three fundamental classes of PDEs (hyperbolic, parabolic and elliptic) by relating the physical characteristics of the problem in question to the message-passing requirement of GNNs. In particular, we investigate the relationship between the physical constants of the equations governing the problem, the spatial and temporal discretisation and the message passing mechanisms in GNNs. When the number of message passing iterations is below these proposed limits, information does not propagate efficiently through the network, resulting in poor solutions, even for deep GNN architectures. In contrast, when the suggested lower bound is satisfied, the GNN parameterisation allows the model to accurately capture the underlying phenomenology, resulting in solvers of adequate accuracy. Examples are provided for four different examples of equations that show the sharpness of the proposed lower bounds.