InfoNCE is a Free Lunch for Semantically guided Graph Contrastive Learning

TL;DR

This paper introduces IFL-GCL, a novel graph contrastive learning method that leverages InfoNCE as a free tool to incorporate semantic guidance, significantly improving pretraining performance for graphs and LLM enhancements.

Contribution

It redefines graph contrastive learning as a positive-unlabeled problem and proves that InfoNCE can be used to extract semantic information effectively.

Findings

Achieves up to 9.05% performance improvement in experiments.

Validates the effectiveness of semantically guided contrastive learning.

Demonstrates significant gains in both IID and OOD scenarios.

Abstract

As an important graph pre-training method, Graph Contrastive Learning (GCL) continues to play a crucial role in the ongoing surge of research on graph foundation models or LLM as enhancer for graphs. Traditional GCL optimizes InfoNCE by using augmentations to define self-supervised tasks, treating augmented pairs as positive samples and others as negative. However, this leads to semantically similar pairs being classified as negative, causing significant sampling bias and limiting performance. In this paper, we argue that GCL is essentially a Positive-Unlabeled (PU) learning problem, where the definition of self-supervised tasks should be semantically guided, i.e., augmented samples with similar semantics are considered positive, while others, with unknown semantics, are treated as unlabeled. From this perspective, the key lies in how to extract semantic information. To achieve this, we propose IFL-GCL, using InfoNCE as a "free lunch" to extract semantic information. Specifically, We first prove that under InfoNCE, the representation similarity of node pairs aligns with the probability that the corresponding contrastive sample is positive. Then we redefine the maximum likelihood objective based on the corrected samples, leading to a new InfoNCE loss function. Extensive experiments on both the graph pretraining framework and LLM as an enhancer show significantly improvements of IFL-GCL in both IID and OOD scenarios, achieving up to a 9.05% improvement, validating the effectiveness of semantically guided. Code for IFL-GCL is publicly available at: https://github.com/Camel-Prince/IFL-GCL.