Contrastive Learning with Synthetic Positives

TL;DR

This paper introduces CLSP, a contrastive learning method that uses synthetic images generated by diffusion models as hard positives, improving SSL performance and establishing a new baseline for synthetic data use.

Contribution

It proposes a novel approach to generate synthetic positive samples via diffusion models, enhancing contrastive learning with diverse, semantically similar images.

Findings

Over 2% improvement in linear evaluation accuracy.

Outperforms previous methods on multiple benchmarks.

Excels in transfer learning across most datasets.

Abstract

Contrastive learning with the nearest neighbor has proved to be one of the most efficient self-supervised learning (SSL) techniques by utilizing the similarity of multiple instances within the same class. However, its efficacy is constrained as the nearest neighbor algorithm primarily identifies "easy" positive pairs, where the representations are already closely located in the embedding space. In this paper, we introduce a novel approach called Contrastive Learning with Synthetic Positives (CLSP) that utilizes synthetic images, generated by an unconditional diffusion model, as the additional positives to help the model learn from diverse positives. Through feature interpolation in the diffusion model sampling process, we generate images with distinct backgrounds yet similar semantic content to the anchor image. These images are considered "hard" positives for the anchor image, and when included as supplementary positives in the contrastive loss, they contribute to a performance improvement of over 2% and 1% in linear evaluation compared to the previous NNCLR and All4One methods across multiple benchmark datasets such as CIFAR10, achieving state-of-the-art methods. On transfer learning benchmarks, CLSP outperforms existing SSL frameworks on 6 out of 8 downstream datasets. We believe CLSP establishes a valuable baseline for future SSL studies incorporating synthetic data in the training process.