GMLP: Building Scalable and Flexible Graph Neural Networks with Feature-Message Passing

TL;DR

GMLP introduces a scalable, flexible graph neural network framework that separates message passing from neural updates, enabling efficient pre-computation and adaptability across different neighborhood levels, achieving state-of-the-art results.

Contribution

The paper proposes GMLP, a novel feature-message passing framework that improves scalability and flexibility of GNNs by decoupling message passing from neural updates.

Findings

GMLP achieves state-of-the-art performance on 11 benchmark datasets.

GMLP demonstrates high training scalability and efficiency on large-scale datasets.

The framework effectively balances performance and computational cost.

Abstract

In recent studies, neural message passing has proved to be an effective way to design graph neural networks (GNNs), which have achieved state-of-the-art performance in many graph-based tasks. However, current neural-message passing architectures typically need to perform an expensive recursive neighborhood expansion in multiple rounds and consequently suffer from a scalability issue. Moreover, most existing neural-message passing schemes are inflexible since they are restricted to fixed-hop neighborhoods and insensitive to the actual demands of different nodes. We circumvent these limitations by a novel feature-message passing framework, called Graph Multi-layer Perceptron (GMLP), which separates the neural update from the message passing. With such separation, GMLP significantly improves the scalability and efficiency by performing the message passing procedure in a pre-compute manner, and is flexible and adaptive in leveraging node feature messages over various levels of localities. We further derive novel variants of scalable GNNs under this framework to achieve the best of both worlds in terms of performance and efficiency. We conduct extensive evaluations on 11 benchmark datasets, including large-scale datasets like ogbn-products and an industrial dataset, demonstrating that GMLP achieves not only the state-of-art performance, but also high training scalability and efficiency.