Similarity-aware Positive Instance Sampling for Graph Contrastive Pre-training

TL;DR

This paper introduces a similarity-aware positive instance sampling method for graph contrastive pre-training, improving the quality of positive samples and enhancing GNN performance across multiple benchmarks.

Contribution

It proposes a novel positive sampling strategy based on domain-specific similarity and hierarchical encoding, addressing issues of illegal or uninformative positive instances in graph contrastive learning.

Findings

Outperforms existing pre-training methods on 13 graph and node classification datasets.

Maintains graph legality and relevance in positive instance selection.

Enhances GNN model performance compared to training from scratch.

Abstract

Graph instance contrastive learning has been proved as an effective task for Graph Neural Network (GNN) pre-training. However, one key issue may seriously impede the representative power in existing works: Positive instances created by current methods often miss crucial information of graphs or even yield illegal instances (such as non-chemically-aware graphs in molecular generation). To remedy this issue, we propose to select positive graph instances directly from existing graphs in the training set, which ultimately maintains the legality and similarity to the target graphs. Our selection is based on certain domain-specific pair-wise similarity measurements as well as sampling from a hierarchical graph encoding similarity relations among graphs. Besides, we develop an adaptive node-level pre-training method to dynamically mask nodes to distribute them evenly in the graph. We conduct extensive experiments on $13$ graph classification and node classification benchmark datasets from various domains. The results demonstrate that the GNN models pre-trained by our strategies can outperform those trained-from-scratch models as well as the variants obtained by existing methods.