Rethinking Rotation in Self-Supervised Contrastive Learning: Adaptive Positive or Negative Data Augmentation

TL;DR

This paper introduces an adaptive data augmentation strategy for contrastive learning that dynamically determines whether rotated images should be treated as positive or negative pairs based on their semantic content, improving learning performance.

Contribution

It proposes PNDA, a novel unsupervised method to adaptively assign rotation-based augmentations as positive or negative pairs in contrastive learning, addressing limitations of fixed treatment.

Findings

PNDA improves contrastive learning performance.

Adaptive augmentation outperforms fixed strategies.

Code available at provided URL.

Abstract

Rotation is frequently listed as a candidate for data augmentation in contrastive learning but seldom provides satisfactory improvements. We argue that this is because the rotated image is always treated as either positive or negative. The semantics of an image can be rotation-invariant or rotation-variant, so whether the rotated image is treated as positive or negative should be determined based on the content of the image. Therefore, we propose a novel augmentation strategy, adaptive Positive or Negative Data Augmentation (PNDA), in which an original and its rotated image are a positive pair if they are semantically close and a negative pair if they are semantically different. To achieve PNDA, we first determine whether rotation is positive or negative on an image-by-image basis in an unsupervised way. Then, we apply PNDA to contrastive learning frameworks. Our experiments showed that PNDA improves the performance of contrastive learning. The code is available at \url{ https://github.com/AtsuMiyai/rethinking_rotation}.