CORE: Contrastive Masked Feature Reconstruction on Graphs

TL;DR

CORE introduces a novel self-supervised learning framework for graphs that combines contrastive learning with masked feature reconstruction, leading to significant improvements in node and graph classification performance.

Contribution

This paper reveals a theoretical connection between MFR and GCL and proposes CORE, a new framework integrating contrastive learning into MFR for enhanced graph representation learning.

Findings

CORE outperforms existing methods like GraphMAE by up to 3.72% on node classification.

CORE achieves state-of-the-art results on graph classification tasks.

Theoretical analysis shows MFR and GCL objectives can converge under certain conditions.

Abstract

In the rapidly evolving field of self-supervised learning on graphs, generative and contrastive methodologies have emerged as two dominant approaches. Our study focuses on masked feature reconstruction (MFR), a generative technique where a model learns to restore the raw features of masked nodes in a self-supervised manner. We observe that both MFR and graph contrastive learning (GCL) aim to maximize agreement between similar elements. Building on this observation, we reveal a novel theoretical insight: under specific conditions, the objectives of MFR and node-level GCL converge, despite their distinct operational mechanisms. This theoretical connection suggests these approaches are complementary rather than fundamentally different, prompting us to explore their integration to enhance self-supervised learning on graphs. Our research presents Contrastive Masked Feature Reconstruction (CORE), a novel graph self-supervised learning framework that integrates contrastive learning into MFR. Specifically, we form positive pairs exclusively between the original and reconstructed features of masked nodes, encouraging the encoder to prioritize contextual information over the node's own features. Additionally, we leverage the masked nodes themselves as negative samples, combining MFR's reconstructive power with GCL's discriminative ability to better capture intrinsic graph structures. Empirically, our proposed framework CORE significantly outperforms MFR across node and graph classification tasks, demonstrating state-of-the-art results. In particular, CORE surpasses GraphMAE and GraphMAE2 by up to 2.80% and 3.72% on node classification tasks, and by up to 3.82% and 3.76% on graph classification tasks.