Heterogeneous Graph Masked Autoencoders

TL;DR

This paper introduces HGMAE, a novel generative self-supervised learning model for heterogeneous graphs that effectively captures complex structures, node attributes, and positional information through innovative masking and training strategies.

Contribution

The paper proposes HGMAE, a new heterogeneous graph masked autoencoder with specialized masking techniques and training strategies to better learn from complex, attribute-rich, and position-aware graph data.

Findings

HGMAE outperforms state-of-the-art baselines on multiple tasks.

Effective encoding of structural, attribute, and positional information.

Demonstrates strong generalization across diverse datasets.

Abstract

Generative self-supervised learning (SSL), especially masked autoencoders, has become one of the most exciting learning paradigms and has shown great potential in handling graph data. However, real-world graphs are always heterogeneous, which poses three critical challenges that existing methods ignore: 1) how to capture complex graph structure? 2) how to incorporate various node attributes? and 3) how to encode different node positions? In light of this, we study the problem of generative SSL on heterogeneous graphs and propose HGMAE, a novel heterogeneous graph masked autoencoder model to address these challenges. HGMAE captures comprehensive graph information via two innovative masking techniques and three unique training strategies. In particular, we first develop metapath masking and adaptive attribute masking with dynamic mask rate to enable effective and stable learning on heterogeneous graphs. We then design several training strategies including metapath-based edge reconstruction to adopt complex structural information, target attribute restoration to incorporate various node attributes, and positional feature prediction to encode node positional information. Extensive experiments demonstrate that HGMAE outperforms both contrastive and generative state-of-the-art baselines on several tasks across multiple datasets. Codes are available at https://github.com/meettyj/HGMAE.