Local Similarity-Aware Deep Feature Embedding

TL;DR

This paper proposes a local similarity-aware deep metric that adapts to local feature structures, improving embedding learning efficiency, convergence speed, and generalization in image retrieval and transfer learning tasks.

Contribution

Introduction of a Position-Dependent Deep Metric (PDDM) unit that learns a local similarity metric, enhancing hard sample mining and embedding performance.

Findings

Faster convergence in deep embedding training.

Improved accuracy on complex image retrieval datasets.

Superior generalization in transfer and zero-shot learning scenarios.

Abstract

Existing deep embedding methods in vision tasks are capable of learning a compact Euclidean space from images, where Euclidean distances correspond to a similarity metric. To make learning more effective and efficient, hard sample mining is usually employed, with samples identified through computing the Euclidean feature distance. However, the global Euclidean distance cannot faithfully characterize the true feature similarity in a complex visual feature space, where the intraclass distance in a high-density region may be larger than the interclass distance in low-density regions. In this paper, we introduce a Position-Dependent Deep Metric (PDDM) unit, which is capable of learning a similarity metric adaptive to local feature structure. The metric can be used to select genuinely hard samples in a local neighborhood to guide the deep embedding learning in an online and robust manner. The new layer is appealing in that it is pluggable to any convolutional networks and is trained end-to-end. Our local similarity-aware feature embedding not only demonstrates faster convergence and boosted performance on two complex image retrieval datasets, its large margin nature also leads to superior generalization results under the large and open set scenarios of transfer learning and zero-shot learning on ImageNet 2010 and ImageNet-10K datasets.