Understanding Contrastive Representation Learning from Positive Unlabeled (PU) Data

TL;DR

This paper introduces a novel contrastive learning framework for positive unlabeled data, improving representation quality and classification accuracy especially when labeled positives are scarce, supported by theoretical analysis and empirical validation.

Contribution

It proposes Positive Unlabeled Contrastive Learning (puCL) and puNCE, new contrastive objectives tailored for PU data, with theoretical guarantees and improved empirical performance.

Findings

Outperforms existing PU learning methods on standard benchmarks

Provides bias-variance analysis and convergence insights

Effective in low-supervision regimes

Abstract

Pretext Invariant Representation Learning (PIRL) followed by Supervised Fine-Tuning (SFT) has become a standard paradigm for learning with limited labels. We extend this approach to the Positive Unlabeled (PU) setting, where only a small set of labeled positives and a large unlabeled pool -- containing both positives and negatives are available. We study this problem under two regimes: (i) without access to the class prior, and (ii) when the prior is known or can be estimated. We introduce Positive Unlabeled Contrastive Learning (puCL), an unbiased and variance reducing contrastive objective that integrates weak supervision from labeled positives judiciously into the contrastive loss. When the class prior is known, we propose Positive Unlabeled InfoNCE (puNCE), a prior-aware extension that re-weights unlabeled samples as soft positive negative mixtures. For downstream classification, we develop a pseudo-labeling algorithm that leverages the structure of the learned embedding space via PU aware clustering. Our framework is supported by theory; offering bias-variance analysis, convergence insights, and generalization guarantees via augmentation concentration; and validated empirically across standard PU benchmarks, where it consistently outperforms existing methods, particularly in low-supervision regimes.