Integrating Prior Knowledge in Contrastive Learning with Kernel

TL;DR

This paper introduces a kernel-based contrastive learning loss that incorporates prior knowledge from generative models or weak attributes, improving representation quality in both natural and medical images.

Contribution

It proposes a novel decoupled uniformity loss that integrates prior knowledge and removes negative-positive coupling in contrastive learning.

Findings

Improves representation quality on natural and medical images.

Outperforms existing contrastive learning methods in weakly supervised scenarios.

Provides theoretical bounds linking contrastive learning to mean embedding theory.

Abstract

Data augmentation is a crucial component in unsupervised contrastive learning (CL). It determines how positive samples are defined and, ultimately, the quality of the learned representation. In this work, we open the door to new perspectives for CL by integrating prior knowledge, given either by generative models -- viewed as prior representations -- or weak attributes in the positive and negative sampling. To this end, we use kernel theory to propose a novel loss, called decoupled uniformity, that i) allows the integration of prior knowledge and ii) removes the negative-positive coupling in the original InfoNCE loss. We draw a connection between contrastive learning and conditional mean embedding theory to derive tight bounds on the downstream classification loss. In an unsupervised setting, we empirically demonstrate that CL benefits from generative models to improve its representation both on natural and medical images. In a weakly supervised scenario, our framework outperforms other unconditional and conditional CL approaches.