Anti-Collapse Loss for Deep Metric Learning Based on Coding Rate Metric

TL;DR

This paper introduces Anti-Collapse Loss, a novel approach in deep metric learning that prevents embedding space collapse by maximizing coding rate, leading to improved discriminative feature representations and better performance.

Contribution

The paper proposes a new loss function based on coding rate principles to prevent feature collapse in deep metric learning, enhancing existing pair- and proxy-based methods.

Findings

Outperforms state-of-the-art methods on benchmark datasets

Effectively prevents embedding space collapse

Improves generalization and feature discrimination

Abstract

Deep metric learning (DML) aims to learn a discriminative high-dimensional embedding space for downstream tasks like classification, clustering, and retrieval. Prior literature predominantly focuses on pair-based and proxy-based methods to maximize inter-class discrepancy and minimize intra-class diversity. However, these methods tend to suffer from the collapse of the embedding space due to their over-reliance on label information. This leads to sub-optimal feature representation and inferior model performance. To maintain the structure of embedding space and avoid feature collapse, we propose a novel loss function called Anti-Collapse Loss. Specifically, our proposed loss primarily draws inspiration from the principle of Maximal Coding Rate Reduction. It promotes the sparseness of feature clusters in the embedding space to prevent collapse by maximizing the average coding rate of sample features or class proxies. Moreover, we integrate our proposed loss with pair-based and proxy-based methods, resulting in notable performance improvement. Comprehensive experiments on benchmark datasets demonstrate that our proposed method outperforms existing state-of-the-art methods. Extensive ablation studies verify the effectiveness of our method in preventing embedding space collapse and promoting generalization performance.