Fast and Distributed Equivariant Graph Neural Networks by Virtual Node Learning

TL;DR

This paper introduces FastEGNN and DistEGNN, innovative equivariant GNNs that efficiently handle large-scale geometric graphs through virtual nodes and distributed learning, achieving high accuracy and scalability.

Contribution

The paper proposes FastEGNN and DistEGNN, novel methods that improve efficiency and scalability of equivariant GNNs for large geometric graphs using virtual nodes and distributed architecture.

Findings

FastEGNN maintains high accuracy on large sparse graphs.

DistEGNN reduces memory and computational costs significantly.

Models outperform existing approaches on multiple large-scale benchmarks.

Abstract

Equivariant Graph Neural Networks (GNNs) have achieved remarkable success across diverse scientific applications. However, existing approaches face critical efficiency challenges when scaling to large geometric graphs and suffer significant performance degradation when the input graphs are sparsified for computational tractability. To address these limitations, we introduce FastEGNN and DistEGNN, two novel enhancements to equivariant GNNs for large-scale geometric graphs. FastEGNN employs a key innovation: a small ordered set of virtual nodes that effectively approximates the large unordered graph of real nodes. Specifically, we implement distinct message passing and aggregation mechanisms for different virtual nodes to ensure mutual distinctiveness, and minimize Maximum Mean Discrepancy (MMD) between virtual and real coordinates to achieve global distributedness. This design enables FastEGNN to maintain high accuracy while efficiently processing large-scale sparse graphs. For extremely large-scale geometric graphs, we present DistEGNN, a distributed extension where virtual nodes act as global bridges between subgraphs in different devices, maintaining consistency while dramatically reducing memory and computational overhead. We comprehensively evaluate our models across four challenging domains: N-body systems (100 nodes), protein dynamics (800 nodes), Water-3D (8,000 nodes), and our new Fluid113K benchmark (113,000 nodes). Results demonstrate superior efficiency and performance, establishing new capabilities in large-scale equivariant graph learning. Code is available at https://github.com/GLAD-RUC/DistEGNN.