Semantics-Consistent Feature Search for Self-Supervised Visual Representation Learning

TL;DR

This paper introduces a semantics-consistent feature search method for self-supervised visual learning, which adaptively focuses on meaningful regions to improve semantic representations and achieves state-of-the-art results.

Contribution

It proposes a novel semantics-consistent feature search (SCFS) method that mitigates semantic inconsistency during augmentation in self-supervised learning.

Findings

SCFS improves downstream task performance.

It enhances semantic focus in learned representations.

Achieves state-of-the-art results on multiple datasets.

Abstract

In contrastive self-supervised learning, the common way to learn discriminative representation is to pull different augmented "views" of the same image closer while pushing all other images further apart, which has been proven to be effective. However, it is unavoidable to construct undesirable views containing different semantic concepts during the augmentation procedure. It would damage the semantic consistency of representation to pull these augmentations closer in the feature space indiscriminately. In this study, we introduce feature-level augmentation and propose a novel semantics-consistent feature search (SCFS) method to mitigate this negative effect. The main idea of SCFS is to adaptively search semantics-consistent features to enhance the contrast between semantics-consistent regions in different augmentations. Thus, the trained model can learn to focus on meaningful object regions, improving the semantic representation ability. Extensive experiments conducted on different datasets and tasks demonstrate that SCFS effectively improves the performance of self-supervised learning and achieves state-of-the-art performance on different downstream tasks.