Localized Contrastive Learning on Graphs

TL;DR

This paper introduces Local-GCL, a simple contrastive learning model for graphs that avoids data augmentation and reduces computational complexity, achieving competitive node representations efficiently.

Contribution

The paper proposes Local-GCL, a novel graph contrastive learning method that uses neighbor-based positives and a kernelized loss for efficiency, without relying on data augmentation.

Findings

Local-GCL performs competitively on various graph datasets.

The kernelized contrastive loss reduces complexity to linear time and space.

The method is effective across different graph scales and properties.

Abstract

Contrastive learning methods based on InfoNCE loss are popular in node representation learning tasks on graph-structured data. However, its reliance on data augmentation and its quadratic computational complexity might lead to inconsistency and inefficiency problems. To mitigate these limitations, in this paper, we introduce a simple yet effective contrastive model named Localized Graph Contrastive Learning (Local-GCL in short). Local-GCL consists of two key designs: 1) We fabricate the positive examples for each node directly using its first-order neighbors, which frees our method from the reliance on carefully-designed graph augmentations; 2) To improve the efficiency of contrastive learning on graphs, we devise a kernelized contrastive loss, which could be approximately computed in linear time and space complexity with respect to the graph size. We provide theoretical analysis to justify the effectiveness and rationality of the proposed methods. Experiments on various datasets with different scales and properties demonstrate that in spite of its simplicity, Local-GCL achieves quite competitive performance in self-supervised node representation learning tasks on graphs with various scales and properties.