Training MLPs on Graphs without Supervision

TL;DR

SimMLP is a novel self-supervised framework that enables MLPs to fully incorporate structural information from graphs, achieving GNN-level performance in various graph learning tasks without relying on neighborhood aggregation.

Contribution

This work introduces SimMLP, the first MLP method capable of matching GNNs' performance by fully integrating structural information through self-supervised learning.

Findings

SimMLP outperforms state-of-the-art baselines on 20 benchmark datasets.

It achieves GNN-level performance in node classification, link prediction, and graph classification.

SimMLP excels in inductive and cold-start scenarios involving unseen nodes.

Abstract

Graph Neural Networks (GNNs) have demonstrated their effectiveness in various graph learning tasks, yet their reliance on neighborhood aggregation during inference poses challenges for deployment in latency-sensitive applications, such as real-time financial fraud detection. To address this limitation, recent studies have proposed distilling knowledge from teacher GNNs into student Multi-Layer Perceptrons (MLPs) trained on node content, aiming to accelerate inference. However, these approaches often inadequately explore structural information when inferring unseen nodes. To this end, we introduce SimMLP, a Self-supervised framework for learning MLPs on graphs, designed to fully integrate rich structural information into MLPs. Notably, SimMLP is the first MLP-learning method that can achieve equivalence to GNNs in the optimal case. The key idea is to employ self-supervised learning to align the representations encoded by graph context-aware GNNs and neighborhood dependency-free MLPs, thereby fully integrating the structural information into MLPs. We provide a comprehensive theoretical analysis, demonstrating the equivalence between SimMLP and GNNs based on mutual information and inductive bias, highlighting SimMLP's advanced structural learning capabilities. Additionally, we conduct extensive experiments on 20 benchmark datasets, covering node classification, link prediction, and graph classification, to showcase SimMLP's superiority over state-of-the-art baselines, particularly in scenarios involving unseen nodes (e.g., inductive and cold-start node classification) where structural insights are crucial. Our codes are available at: https://github.com/Zehong-Wang/SimMLP.