On Measuring Long-Range Interactions in Graph Neural Networks

TL;DR

This paper formalizes the concept of long-range interactions in graph neural networks, introduces a range measure for operators, and evaluates existing architectures and tasks to better understand and address the long-range problem.

Contribution

It provides a theoretical framework and a quantitative measure for long-range interactions in graph tasks, bridging the gap between empirical evaluation and formal understanding.

Findings

Introduces a range measure for graph operators.

Validates the measure with synthetic experiments.

Analyzes existing tasks and architectures for long-range capabilities.

Abstract

Long-range graph tasks -- those dependent on interactions between distant nodes -- are an open problem in graph neural network research. Real-world benchmark tasks, especially the Long Range Graph Benchmark, have become popular for validating the long-range capability of proposed architectures. However, this is an empirical approach that lacks both robustness and theoretical underpinning; a more principled characterization of the long-range problem is required. To bridge this gap, we formalize long-range interactions in graph tasks, introduce a range measure for operators on graphs, and validate it with synthetic experiments. We then leverage our measure to examine commonly used tasks and architectures, and discuss to what extent they are, in fact, long-range. We believe our work advances efforts to define and address the long-range problem on graphs, and that our range measure will aid evaluation of new datasets and architectures.