Neighbor2Seq: Deep Learning on Massive Graphs by Transforming Neighbors to Sequences

TL;DR

Neighbor2Seq transforms node neighborhoods into sequences, enabling efficient deep learning on massive graphs by leveraging grid-like data operations, thus overcoming GNN scalability issues.

Contribution

It introduces a novel sequence transformation of neighborhoods, allowing scalable deep learning on large graphs with improved efficiency and performance.

Findings

Scalable to graphs with over 111 million nodes.

Achieves superior performance on massive and medium-scale graphs.

Enables deep learning operations on massive graphs efficiently.

Abstract

Modern graph neural networks (GNNs) use a message passing scheme and have achieved great success in many fields. However, this recursive design inherently leads to excessive computation and memory requirements, making it not applicable to massive real-world graphs. In this work, we propose the Neighbor2Seq to transform the hierarchical neighborhood of each node into a sequence. This novel transformation enables the subsequent mini-batch training for general deep learning operations, such as convolution and attention, that are designed for grid-like data and are shown to be powerful in various domains. Therefore, our Neighbor2Seq naturally endows GNNs with the efficiency and advantages of deep learning operations on grid-like data by precomputing the Neighbor2Seq transformations. We evaluate our method on a massive graph, with more than 111 million nodes and 1.6 billion edges, as well as several medium-scale graphs. Results show that our proposed method is scalable to massive graphs and achieves superior performance across massive and medium-scale graphs. Our code is available at https://github.com/divelab/Neighbor2Seq.