SimO Loss: Anchor-Free Contrastive Loss for Fine-Grained Supervised Contrastive Learning

TL;DR

This paper introduces SimO loss, an anchor-free contrastive loss function that enhances fine-grained supervised contrastive learning by creating orthogonal, class-specific neighborhoods in the embedding space.

Contribution

The paper proposes a novel SimO loss for anchor-free contrastive learning, promoting orthogonality and class separation in embeddings, with validation on CIFAR-10.

Findings

Creates orthogonal class neighborhoods in embeddings

Balances intra-class variability with class separation

Demonstrates effectiveness on CIFAR-10 dataset

Abstract

We introduce a novel anchor-free contrastive learning (AFCL) method leveraging our proposed Similarity-Orthogonality (SimO) loss. Our approach minimizes a semi-metric discriminative loss function that simultaneously optimizes two key objectives: reducing the distance and orthogonality between embeddings of similar inputs while maximizing these metrics for dissimilar inputs, facilitating more fine-grained contrastive learning. The AFCL method, powered by SimO loss, creates a fiber bundle topological structure in the embedding space, forming class-specific, internally cohesive yet orthogonal neighborhoods. We validate the efficacy of our method on the CIFAR-10 dataset, providing visualizations that demonstrate the impact of SimO loss on the embedding space. Our results illustrate the formation of distinct, orthogonal class neighborhoods, showcasing the method's ability to create well-structured embeddings that balance class separation with intra-class variability. This work opens new avenues for understanding and leveraging the geometric properties of learned representations in various machine learning tasks.