Hi-GMAE: Hierarchical Graph Masked Autoencoders

TL;DR

Hi-GMAE introduces a hierarchical, multi-scale masked autoencoder framework for graphs that effectively captures complex hierarchical structures, outperforming existing single-scale models across multiple datasets.

Contribution

The paper proposes Hi-GMAE, a novel multi-scale GMAE with a hierarchical graph construction and a coarse-to-fine masking strategy, addressing limitations of single-scale models.

Findings

Outperforms 29 state-of-the-art self-supervised methods

Effective in capturing hierarchical graph information

Demonstrates consistent improvements across 17 datasets

Abstract

Graph Masked Autoencoders (GMAEs) have emerged as a notable self-supervised learning approach for graph-structured data. Existing GMAE models primarily focus on reconstructing node-level information, categorizing them as single-scale GMAEs. This methodology, while effective in certain contexts, tends to overlook the complex hierarchical structures inherent in many real-world graphs. For instance, molecular graphs exhibit a clear hierarchical organization in the form of the atoms-functional groups-molecules structure. Therefore, the inability of single-scale GMAE models to incorporate these hierarchical relationships often results in an inadequate capture of crucial high-level graph information, leading to a noticeable decline in performance. To address this limitation, we propose Hierarchical Graph Masked AutoEncoders (Hi-GMAE), a novel multi-scale GMAE framework designed to handle the hierarchical structures within graphs. First, Hi-GMAE constructs a multi-scale graph hierarchy through graph pooling, enabling the exploration of graph structures across different granularity levels. To ensure masking uniformity of subgraphs across these scales, we propose a novel coarse-to-fine strategy that initiates masking at the coarsest scale and progressively back-projects the mask to finer scales. Furthermore, we integrate a gradual recovery strategy with the masking process to mitigate the learning challenges posed by completely masked subgraphs. Our experiments on 17 graph datasets, covering two graph learning tasks, consistently demonstrate that Hi-GMAE outperforms 29 state-of-the-art self-supervised competitors in capturing comprehensive graph information.