Beyond Instance Discrimination: Relation-aware Contrastive Self-supervised Learning

TL;DR

This paper introduces ReCo, a relation-aware contrastive self-supervised learning method that models instance relations to improve semantic structure retention and achieves superior performance on benchmarks.

Contribution

ReCo integrates global distribution and local interpolation relations into contrastive learning, enhancing semantic understanding and feature space structuring.

Findings

ReCo outperforms baseline CSL methods on standard benchmarks.

Explicit relation modeling prevents semantically similar samples from being pushed apart.

ReCo achieves consistent performance improvements across various datasets.

Abstract

Contrastive self-supervised learning (CSL) based on instance discrimination typically attracts positive samples while repelling negatives to learn representations with pre-defined binary self-supervision. However, vanilla CSL is inadequate in modeling sophisticated instance relations, limiting the learned model to retain fine semantic structure. On the one hand, samples with the same semantic category are inevitably pushed away as negatives. On the other hand, differences among samples cannot be captured. In this paper, we present relation-aware contrastive self-supervised learning (ReCo) to integrate instance relations, i.e., global distribution relation and local interpolation relation, into the CSL framework in a plug-and-play fashion. Specifically, we align similarity distributions calculated between the positive anchor views and the negatives at the global level to exploit diverse similarity relations among instances. Local-level interpolation consistency between the pixel space and the feature space is applied to quantitatively model the feature differences of samples with distinct apparent similarities. Through explicitly instance relation modeling, our ReCo avoids irrationally pushing away semantically identical samples and carves a well-structured feature space. Extensive experiments conducted on commonly used benchmarks justify that our ReCo consistently gains remarkable performance improvements.