UGMAE: A Unified Framework for Graph Masked Autoencoders

TL;DR

UGMAE introduces a comprehensive framework for graph masked autoencoders that adaptively masks nodes, captures holistic graph info, encodes semantic knowledge, and stabilizes reconstructions, significantly improving graph representation learning.

Contribution

The paper presents UGMAE, a unified framework that addresses key limitations of existing graph masked autoencoders through novel modules for adaptivity, integrity, complementarity, and consistency.

Findings

Outperforms state-of-the-art methods on multiple graph tasks.

Effectively captures semantic and topological information.

Enhances stability and robustness of reconstructions.

Abstract

Generative self-supervised learning on graphs, particularly graph masked autoencoders, has emerged as a popular learning paradigm and demonstrated its efficacy in handling non-Euclidean data. However, several remaining issues limit the capability of existing methods: 1) the disregard of uneven node significance in masking, 2) the underutilization of holistic graph information, 3) the ignorance of semantic knowledge in the representation space due to the exclusive use of reconstruction loss in the output space, and 4) the unstable reconstructions caused by the large volume of masked contents. In light of this, we propose UGMAE, a unified framework for graph masked autoencoders to address these issues from the perspectives of adaptivity, integrity, complementarity, and consistency. Specifically, we first develop an adaptive feature mask generator to account for the unique significance of nodes and sample informative masks (adaptivity). We then design a ranking-based structure reconstruction objective joint with feature reconstruction to capture holistic graph information and emphasize the topological proximity between neighbors (integrity). After that, we present a bootstrapping-based similarity module to encode the high-level semantic knowledge in the representation space, complementary to the low-level reconstruction in the output space (complementarity). Finally, we build a consistency assurance module to provide reconstruction objectives with extra stabilized consistency targets (consistency). Extensive experiments demonstrate that UGMAE outperforms both contrastive and generative state-of-the-art baselines on several tasks across multiple datasets.