Graph Positional Autoencoders as Self-supervised Learners

TL;DR

This paper introduces Graph Positional Autoencoders (GraphPAE), a novel self-supervised learning method for graphs that captures diverse structural signals and outperforms existing approaches across multiple tasks.

Contribution

GraphPAE employs a dual-path architecture to reconstruct node features and positions, effectively capturing high-frequency structural information beyond traditional masking methods.

Findings

Achieves state-of-the-art results in heterophilic node classification.

Outperforms baselines in graph property prediction.

Demonstrates strong transfer learning capabilities.

Abstract

Graph self-supervised learning seeks to learn effective graph representations without relying on labeled data. Among various approaches, graph autoencoders (GAEs) have gained significant attention for their efficiency and scalability. Typically, GAEs take incomplete graphs as input and predict missing elements, such as masked nodes or edges. While effective, our experimental investigation reveals that traditional node or edge masking paradigms primarily capture low-frequency signals in the graph and fail to learn the expressive structural information. To address these issues, we propose Graph Positional Autoencoders (GraphPAE), which employs a dual-path architecture to reconstruct both node features and positions. Specifically, the feature path uses positional encoding to enhance the message-passing processing, improving GAE's ability to predict the corrupted information. The position path, on the other hand, leverages node representations to refine positions and approximate eigenvectors, thereby enabling the encoder to learn diverse frequency information. We conduct extensive experiments to verify the effectiveness of GraphPAE, including heterophilic node classification, graph property prediction, and transfer learning. The results demonstrate that GraphPAE achieves state-of-the-art performance and consistently outperforms baselines by a large margin.