SpeGCL: Self-supervised Graph Spectrum Contrastive Learning without Positive Samples

TL;DR

SpeGCL introduces a spectral contrastive learning framework that leverages high-frequency information and solely uses negative samples, improving graph representation learning without positive pairs.

Contribution

The paper proposes SpeGCL, a novel spectral GCL method that utilizes Fourier transforms and negative samples only, addressing limitations of existing GCL approaches.

Findings

SpeGCL outperforms state-of-the-art GCL methods in various learning settings.

Utilizing high-frequency information accelerates model convergence.

Negative sample-based optimization is theoretically justified and effective.

Abstract

Graph Contrastive Learning (GCL) excels at managing noise and fluctuations in input data, making it popular in various fields (e.g., social networks, and knowledge graphs). Our study finds that the difference in high-frequency information between augmented graphs is greater than that in low-frequency information. However, most existing GCL methods focus mainly on the time domain (low-frequency information) for node feature representations and cannot make good use of high-frequency information to speed up model convergence. Furthermore, existing GCL paradigms optimize graph embedding representations by pulling the distance between positive sample pairs closer and pushing the distance between positive and negative sample pairs farther away, but our theoretical analysis shows that graph contrastive learning benefits from pushing negative pairs farther away rather than pulling positive pairs closer. To solve the above-mentioned problems, we propose a novel spectral GCL framework without positive samples, named SpeGCL. Specifically, to solve the problem that existing GCL methods cannot utilize high-frequency information, SpeGCL uses a Fourier transform to extract high-frequency and low-frequency information of node features, and constructs a contrastive learning mechanism in a Fourier space to obtain better node feature representation. Furthermore, SpeGCL relies entirely on negative samples to refine the graph embedding. We also provide a theoretical justification for the efficacy of using only negative samples in SpeGCL. Extensive experiments on un-supervised learning, transfer learning, and semi-supervised learning have validated the superiority of our SpeGCL framework over the state-of-the-art GCL methods.