An Augmentation-Aware Theory for Self-Supervised Contrastive Learning

TL;DR

This paper introduces an augmentation-aware theoretical framework for self-supervised contrastive learning, revealing how data augmentation influences learning performance and providing insights verified through experiments.

Contribution

It proposes the first augmentation-aware error bound for contrastive learning, explicitly linking data augmentation types to the learning risk.

Findings

Data augmentation significantly impacts the error bound in contrastive learning.

Certain augmentation methods can tighten or loosen the error bound.

Experimental results validate the theoretical predictions.

Abstract

Self-supervised contrastive learning has emerged as a powerful tool in machine learning and computer vision to learn meaningful representations from unlabeled data. Meanwhile, its empirical success has encouraged many theoretical studies to reveal the learning mechanisms. However, in the existing theoretical research, the role of data augmentation is still under-exploited, especially the effects of specific augmentation types. To fill in the blank, we for the first time propose an augmentation-aware error bound for self-supervised contrastive learning, showing that the supervised risk is bounded not only by the unsupervised risk, but also explicitly by a trade-off induced by data augmentation. Then, under a novel semantic label assumption, we discuss how certain augmentation methods affect the error bound. Lastly, we conduct both pixel- and representation-level experiments to verify our proposed theoretical results.