Constructive Assimilation: Boosting Contrastive Learning Performance through View Generation Strategies

TL;DR

This paper introduces a method to combine learned view generation with expert transformations in contrastive learning, significantly improving performance across multiple datasets.

Contribution

It proposes a novel approach to assimilate generated views with expert transformations, enhancing contrastive learning effectiveness.

Findings

Up to 3.6% performance improvement on three datasets

Systematic analysis of view generation and assimilation methods

Constructive assimilation outperforms replacing expert transformations

Abstract

Transformations based on domain expertise (expert transformations), such as random-resized-crop and color-jitter, have proven critical to the success of contrastive learning techniques such as SimCLR. Recently, several attempts have been made to replace such domain-specific, human-designed transformations with generated views that are learned. However for imagery data, so far none of these view-generation methods has been able to outperform expert transformations. In this work, we tackle a different question: instead of replacing expert transformations with generated views, can we constructively assimilate generated views with expert transformations? We answer this question in the affirmative and propose a view generation method and a simple, effective assimilation method that together improve the state-of-the-art by up to ~3.6% on three different datasets. Importantly, we conduct a detailed empirical study that systematically analyzes a range of view generation and assimilation methods and provides a holistic picture of the efficacy of learned views in contrastive representation learning.