Extending the RANGE of Graph Neural Networks: Relaying Attention Nodes for Global Encoding

TL;DR

This paper introduces RANGE, a novel attention-based framework for graph neural networks that effectively captures long-range interactions in large systems, overcoming local information bottlenecks with minimal computational overhead.

Contribution

RANGE is the first virtual-node message-passing method integrating attention, positional encodings, and regularization to enhance long-range information flow in GNNs.

Findings

Significantly reduces oversquashing effects in GNNs.

Achieves high accuracy in modeling long-range interactions.

Operates with negligible additional computational cost.

Abstract

Graph Neural Networks (GNNs) are routinely used in molecular physics, social sciences, and economics to model many-body interactions in graph-like systems. However, GNNs are inherently local and can suffer from information flow bottlenecks. This is particularly problematic when modeling large molecular systems, where dispersion forces and local electric field variations drive collective structural changes. Existing solutions face challenges related to computational cost and scalability. We introduce RANGE, a model-agnostic framework that employs an attention-based aggregation-broadcast mechanism that significantly reduces oversquashing effects, and achieves remarkable accuracy in capturing long-range interactions at a negligible computational cost. Notably, RANGE is the first virtual-node message-passing implementation to integrate attention with positional encodings and regularization to dynamically expand virtual representations. This work lays the foundation for next-generation of machine-learned force fields, offering accurate and efficient modeling of long-range interactions for simulating large molecular systems.