Introspective Deep Metric Learning for Image Retrieval

TL;DR

This paper introduces an introspective deep metric learning framework that incorporates uncertainty modeling to improve image retrieval robustness and achieves state-of-the-art results on multiple datasets.

Contribution

It proposes a novel IDML framework that uses both semantic and uncertainty embeddings for more reliable image similarity comparisons.

Findings

Achieves state-of-the-art results on CUB-200-2011, Cars196, and Stanford Online Products datasets.

Demonstrates improved robustness in image retrieval and clustering tasks.

Provides an in-depth analysis validating the effectiveness of uncertainty modeling.

Abstract

This paper proposes an introspective deep metric learning (IDML) framework for uncertainty-aware comparisons of images. Conventional deep metric learning methods produce confident semantic distances between images regardless of the uncertainty level. However, we argue that a good similarity model should consider the semantic discrepancies with caution to better deal with ambiguous images for more robust training. To achieve this, we propose to represent an image using not only a semantic embedding but also an accompanying uncertainty embedding, which describes the semantic characteristics and ambiguity of an image, respectively. We further propose an introspective similarity metric to make similarity judgments between images considering both their semantic differences and ambiguities. The proposed IDML framework improves the performance of deep metric learning through uncertainty modeling and attains state-of-the-art results on the widely used CUB-200-2011, Cars196, and Stanford Online Products datasets for image retrieval and clustering. We further provide an in-depth analysis of our framework to demonstrate the effectiveness and reliability of IDML. Code is available at: https://github.com/wzzheng/IDML.