Deep Metric Learning with Soft Orthogonal Proxies

TL;DR

This paper introduces a novel Soft Orthogonality constraint for proxies in Deep Metric Learning, improving class separation and reducing redundancy, leading to superior image retrieval performance.

Contribution

The paper proposes a Soft Orthogonality regularization for proxies in DML, enhancing their distribution and improving retrieval accuracy.

Findings

Outperforms state-of-the-art methods on four benchmarks

Proxies become more orthogonal and less correlated

Improved convergence and retrieval accuracy

Abstract

Deep Metric Learning (DML) models rely on strong representations and similarity-based measures with specific loss functions. Proxy-based losses have shown great performance compared to pair-based losses in terms of convergence speed. However, proxies that are assigned to different classes may end up being closely located in the embedding space and hence having a hard time to distinguish between positive and negative items. Alternatively, they may become highly correlated and hence provide redundant information with the model. To address these issues, we propose a novel approach that introduces Soft Orthogonality (SO) constraint on proxies. The constraint ensures the proxies to be as orthogonal as possible and hence control their positions in the embedding space. Our approach leverages Data-Efficient Image Transformer (DeiT) as an encoder to extract contextual features from images along with a DML objective. The objective is made of the Proxy Anchor loss along with the SO regularization. We evaluate our method on four public benchmarks for category-level image retrieval and demonstrate its effectiveness with comprehensive experimental results and ablation studies. Our evaluations demonstrate the superiority of our proposed approach over state-of-the-art methods by a significant margin.