Feature Augmentation for Self-supervised Contrastive Learning: A Closer Look

TL;DR

This paper introduces a feature augmentation framework for self-supervised contrastive learning that enhances data diversity in feature space, leading to improved generalization and robustness across image classification and object detection tasks.

Contribution

It proposes a domain-agnostic feature augmentation method, systematically investigates its architectures and principles, and demonstrates consistent performance improvements in pre-trained models.

Findings

Feature augmentation improves data diversity and model robustness.

Integrating feature augmentation enhances downstream task performance.

Practical principles for effective feature augmentation are identified.

Abstract

Self-supervised contrastive learning heavily relies on the view variance brought by data augmentation, so that it can learn a view-invariant pre-trained representation. Beyond increasing the view variance for contrast, this work focuses on improving the diversity of training data, to improve the generalization and robustness of the pre-trained models. To this end, we propose a unified framework to conduct data augmentation in the feature space, known as feature augmentation. This strategy is domain-agnostic, which augments similar features to the original ones and thus improves the data diversity. We perform a systematic investigation of various feature augmentation architectures, the gradient-flow skill, and the relationship between feature augmentation and traditional data augmentation. Our study reveals some practical principles for feature augmentation in self-contrastive learning. By integrating feature augmentation on the instance discrimination or the instance similarity paradigm, we consistently improve the performance of pre-trained feature learning and gain better generalization over the downstream image classification and object detection task.