Multi-Head Deep Metric Learning Using Global and Local Representations

TL;DR

This paper introduces a novel deep metric learning approach that combines pairwise and proxy loss functions with global and local features, enhanced by second-order attention, achieving state-of-the-art results.

Contribution

The paper presents a hybrid loss function integrating pairwise and proxy-based losses with global and local features, along with second-order attention for improved retrieval.

Findings

Achieved state-of-the-art performance on four benchmarks.

Hybrid loss accelerates convergence and enhances data relations.

Second-order attention improves feature representation.

Abstract

Deep Metric Learning (DML) models often require strong local and global representations, however, effective integration of local and global features in DML model training is a challenge. DML models are often trained with specific loss functions, including pairwise-based and proxy-based losses. The pairwise-based loss functions leverage rich semantic relations among data points, however, they often suffer from slow convergence during DML model training. On the other hand, the proxy-based loss functions often lead to significant speedups in convergence during training, while the rich relations among data points are often not fully explored by the proxy-based losses. In this paper, we propose a novel DML approach to address these challenges. The proposed DML approach makes use of a hybrid loss by integrating the pairwise-based and the proxy-based loss functions to leverage rich data-to-data relations as well as fast convergence. Furthermore, the proposed DML approach utilizes both global and local features to obtain rich representations in DML model training. Finally, we also use the second-order attention for feature enhancement to improve accurate and efficient retrieval. In our experiments, we extensively evaluated the proposed DML approach on four public benchmarks, and the experimental results demonstrate that the proposed method achieved state-of-the-art performance on all benchmarks.