Graph Attention Multi-Layer Perceptron

TL;DR

The paper introduces GAMLP, a scalable graph neural network architecture that captures multi-scale graph information, achieving state-of-the-art results and high efficiency on large-scale industrial datasets.

Contribution

Proposes GAMLP, a novel GNN architecture that addresses over-smoothing and scalability issues by capturing correlations across multiple graph scales.

Findings

Outperforms GAT by 1.3% accuracy on Tencent Video dataset

Achieves up to 50x training speedup

Ranks top-1 on OGB large-scale graph benchmarks

Abstract

Graph neural networks (GNNs) have achieved great success in many graph-based applications. However, the enormous size and high sparsity level of graphs hinder their applications under industrial scenarios. Although some scalable GNNs are proposed for large-scale graphs, they adopt a fixed $K$-hop neighborhood for each node, thus facing the over-smoothing issue when adopting large propagation depths for nodes within sparse regions. To tackle the above issue, we propose a new GNN architecture -- Graph Attention Multi-Layer Perceptron (GAMLP), which can capture the underlying correlations between different scales of graph knowledge. We have deployed GAMLP in Tencent with the Angel platform, and we further evaluate GAMLP on both real-world datasets and large-scale industrial datasets. Extensive experiments on these 14 graph datasets demonstrate that GAMLP achieves state-of-the-art performance while enjoying high scalability and efficiency. Specifically, it outperforms GAT by 1.3\% regarding predictive accuracy on our large-scale Tencent Video dataset while achieving up to $50\times$ training speedup. Besides, it ranks top-1 on both the leaderboards of the largest homogeneous and heterogeneous graph (i.e., ogbn-papers100M and ogbn-mag) of Open Graph Benchmark.