Directional Self-supervised Learning for Heavy Image Augmentations

TL;DR

This paper introduces DSSL, a novel self-supervised learning approach that leverages heavy augmentations by treating augmented views as a partially ordered set, improving robustness and compatibility with existing frameworks.

Contribution

DSSL is a new paradigm that incorporates directional relationships among augmented views, enabling the use of more diverse augmentations in self-supervised learning.

Findings

Improves performance across CIFAR and ImageNet datasets.

Compatible with popular self-supervised frameworks like SimCLR, SimSiam, BYOL.

Enhances robustness by utilizing heavier augmentations.

Abstract

Despite the large augmentation family, only a few cherry-picked robust augmentation policies are beneficial to self-supervised image representation learning. In this paper, we propose a directional self-supervised learning paradigm (DSSL), which is compatible with significantly more augmentations. Specifically, we adapt heavy augmentation policies after the views lightly augmented by standard augmentations, to generate harder view (HV). HV usually has a higher deviation from the original image than the lightly augmented standard view (SV). Unlike previous methods equally pairing all augmented views to symmetrically maximize their similarities, DSSL treats augmented views of the same instance as a partially ordered set (with directions as SV$\leftrightarrow $SV, SV$\leftarrow$HV), and then equips a directional objective function respecting to the derived relationships among views. DSSL can be easily implemented with a few lines of codes and is highly flexible to popular self-supervised learning frameworks, including SimCLR, SimSiam, BYOL. Extensive experimental results on CIFAR and ImageNet demonstrated that DSSL can stably improve various baselines with compatibility to a wider range of augmentations.