Spacing Loss for Discovering Novel Categories

TL;DR

This paper introduces Spacing Loss, a novel loss function for Novel Class Discovery that enhances class separability in the latent space, applicable as a standalone or supplementary method, validated on CIFAR datasets.

Contribution

The paper proposes Spacing Loss, a new loss function for NCD that improves class separation and can be integrated into existing approaches.

Findings

Spacing Loss improves class separability in latent space.

The method enhances existing NCD approaches.

Experimental results on CIFAR datasets validate effectiveness.

Abstract

Novel Class Discovery (NCD) is a learning paradigm, where a machine learning model is tasked to semantically group instances from unlabeled data, by utilizing labeled instances from a disjoint set of classes. In this work, we first characterize existing NCD approaches into single-stage and two-stage methods based on whether they require access to labeled and unlabeled data together while discovering new classes. Next, we devise a simple yet powerful loss function that enforces separability in the latent space using cues from multi-dimensional scaling, which we refer to as Spacing Loss. Our proposed formulation can either operate as a standalone method or can be plugged into existing methods to enhance them. We validate the efficacy of Spacing Loss with thorough experimental evaluation across multiple settings on CIFAR-10 and CIFAR-100 datasets.