Partitioning Image Representation in Contrastive Learning

TL;DR

This paper introduces a partitioned representation in contrastive learning that separates common class features from individual sample features, improving data representation and downstream task performance.

Contribution

It proposes a novel partitioned representation with content and style parts, decomposing contrastive loss to learn both shared and unique features.

Findings

Successfully separates content and style features in VAE framework

Outperforms conventional BYOL in linear separability

Enhances few-shot learning performance

Abstract

In contrastive learning in the image domain, the anchor and positive samples are forced to have as close representations as possible. However, forcing the two samples to have the same representation could be misleading because the data augmentation techniques make the two samples different. In this paper, we introduce a new representation, partitioned representation, which can learn both common and unique features of the anchor and positive samples in contrastive learning. The partitioned representation consists of two parts: the content part and the style part. The content part represents common features of the class, and the style part represents the own features of each sample, which can lead to the representation of the data augmentation method. We can achieve the partitioned representation simply by decomposing a loss function of contrastive learning into two terms on the two separate representations, respectively. To evaluate our representation with two parts, we take two framework models: Variational AutoEncoder (VAE) and BootstrapYour Own Latent(BYOL) to show the separability of content and style, and to confirm the generalization ability in classification, respectively. Based on the experiments, we show that our approach can separate two types of information in the VAE framework and outperforms the conventional BYOL in linear separability and a few-shot learning task as downstream tasks.