Beyond Augmentation: Leveraging Inter-Instance Relation in Self-Supervised Representation Learning

TL;DR

This paper proposes a graph-based method for self-supervised learning that captures inter-instance relationships using KNN graphs and GNNs, leading to improved accuracy on multiple datasets.

Contribution

It introduces a novel approach integrating graph theory into self-supervised learning to leverage inter-instance relationships during pretraining.

Findings

Achieves 7.3% accuracy improvement on CIFAR-10

Improves 3.2% on ImageNet-100

Gains 1.0% on ImageNet-1K

Abstract

This paper introduces a novel approach that integrates graph theory into self-supervised representation learning. Traditional methods focus on intra-instance variations generated by applying augmentations. However, they often overlook important inter-instance relationships. While our method retains the intra-instance property, it further captures inter-instance relationships by constructing k-nearest neighbor (KNN) graphs for both teacher and student streams during pretraining. In these graphs, nodes represent samples along with their latent representations. Edges encode the similarity between instances. Following pretraining, a representation refinement phase is performed. In this phase, Graph Neural Networks (GNNs) propagate messages not only among immediate neighbors but also across multiple hops, thereby enabling broader contextual integration. Experimental results on CIFAR-10, ImageNet-100, and ImageNet-1K demonstrate accuracy improvements of 7.3%, 3.2%, and 1.0%, respectively, over state-of-the-art methods. These results highlight the effectiveness of the proposed graph based mechanism. The code is publicly available at https://github.com/alijavidani/SSL-GraphNNCLR.