Silhouette Loss: Differentiable Global Structure Learning for Deep Representations

TL;DR

This paper introduces Soft Silhouette Loss, a differentiable clustering-inspired objective that enhances deep representations by explicitly optimizing global structure, and demonstrates its effectiveness when combined with cross-entropy and contrastive learning.

Contribution

The paper proposes a novel Soft Silhouette Loss that captures global structure in deep embedding spaces and can be integrated with existing loss functions for improved performance.

Findings

Augmenting cross-entropy with Soft Silhouette Loss improves accuracy.

Hybrid of Soft Silhouette Loss and supervised contrastive learning outperforms individual methods.

Combined approach achieves higher accuracy with lower computational overhead.

Abstract

Learning discriminative representations is a central goal of supervised deep learning. While cross-entropy (CE) remains the dominant objective for classification, it does not explicitly enforce desirable geometric properties in the embedding space, such as intra-class compactness and inter-class separation. Existing metric learning approaches, including supervised contrastive learning (SupCon) and proxy-based methods, address this limitation by operating on pairwise or proxy-based relationships, but often increase computational cost and complexity. In this work, we introduce Soft Silhouette Loss, a novel differentiable objective inspired by the classical silhouette coefficient from clustering analysis. Unlike pairwise objectives, our formulation evaluates each sample against all classes in the batch, providing a batch-level notion of global structure. The proposed loss directly encourages samples to be closer to their own class than to competing classes, while remaining lightweight. Soft Silhouette Loss can be seamlessly combined with cross-entropy, and is also complementary to supervised contrastive learning. We propose a hybrid objective that integrates them, jointly optimizing local pairwise consistency and global cluster structure. Extensive experiments on seven diverse datasets demonstrate that: (i) augmenting CE with Soft Silhouette Loss consistently improves over CE and other metric learning baselines; (ii) the hybrid formulation outperforms SupCon alone; and (iii) the combined method achieves the best performance, improving average top-1 accuracy from 36.71% (CE) and 37.85% (SupCon2) to 39.08%, while incurring substantially lower computational overhead. These results suggest that classical clustering principles can be reinterpreted as differentiable objectives for deep learning, enabling efficient optimization of both local and global structure in representation spaces.