Recipe for a General, Powerful, Scalable Graph Transformer
Ladislav Ramp\'a\v{s}ek, Mikhail Galkin, Vijay Prakash Dwivedi, Anh, Tuan Luu, Guy Wolf, Dominique Beaini

TL;DR
This paper introduces GraphGPS, a modular, scalable graph Transformer architecture with linear complexity that achieves state-of-the-art results across diverse benchmarks by combining local and global mechanisms.
Contribution
It presents a novel, modular framework for graph Transformers with linear complexity, unifying various encoding strategies and demonstrating superior scalability and performance.
Findings
Achieves state-of-the-art results on 16 benchmarks
Supports multiple encoding types within a unified framework
Demonstrates efficiency and scalability on large graphs
Abstract
We propose a recipe on how to build a general, powerful, scalable (GPS) graph Transformer with linear complexity and state-of-the-art results on a diverse set of benchmarks. Graph Transformers (GTs) have gained popularity in the field of graph representation learning with a variety of recent publications but they lack a common foundation about what constitutes a good positional or structural encoding, and what differentiates them. In this paper, we summarize the different types of encodings with a clearer definition and categorize them as being , or . The prior GTs are constrained to small graphs with a few hundred nodes, here we propose the first architecture with a complexity linear in the number of nodes and edges by decoupling the local real-edge aggregation from the fully-connected Transformer. We argue that this…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsAdvanced Graph Neural Networks · Data Quality and Management
MethodsLaplacian EigenMap · Greedy Policy Search · Laplacian Positional Encodings · Goal-Driven Tree-Structured Neural Model · Graph Transformer · Performer · Message Passing Neural Network · Transformer
