SinSim: Sinkhorn-Regularized SimCLR

TL;DR

SinSim enhances contrastive self-supervised learning by integrating Sinkhorn regularization, resulting in more structured and discriminative feature representations that outperform standard SimCLR and are competitive with other advanced methods.

Contribution

We introduce SinSim, which incorporates Sinkhorn regularization into SimCLR, providing a novel way to enforce global structure in learned representations.

Findings

SinSim outperforms SimCLR on multiple datasets.

Visualizations show improved class separation.

Comparable performance to VICReg and Barlow Twins.

Abstract

Self-supervised learning has revolutionized representation learning by eliminating the need for labeled data. Contrastive learning methods, such as SimCLR, maximize the agreement between augmented views of an image but lack explicit regularization to enforce a globally structured latent space. This limitation often leads to suboptimal generalization. We propose SinSim, a novel extension of SimCLR that integrates Sinkhorn regularization from optimal transport theory to enhance representation structure. The Sinkhorn loss, an entropy-regularized Wasserstein distance, encourages a well-dispersed and geometry-aware feature space, preserving discriminative power. Empirical evaluations on various datasets demonstrate that SinSim outperforms SimCLR and achieves competitive performance against prominent self-supervised methods such as VICReg and Barlow Twins. UMAP visualizations further reveal improved class separability and structured feature distributions. These results indicate that integrating optimal transport regularization into contrastive learning provides a principled and effective mechanism for learning robust, well-structured representations. Our findings open new directions for applying transport-based constraints in self-supervised learning frameworks.