Catching Image Retrieval Generalization

TL;DR

This paper identifies limitations of Recall@K in estimating image retrieval generalization and proposes a new metric that better assesses how well models perform on unseen data, providing deeper insights into deep metric learning.

Contribution

The paper introduces a novel retrieval performance metric that accurately estimates generalization, addressing the dependence of Recall@K on dataset class count.

Findings

Recall@K depends on dataset class count

The new metric better estimates generalization in image retrieval

Insights into deep metric learning performance

Abstract

The concepts of overfitting and generalization are vital for evaluating machine learning models. In this work, we show that the popular Recall@K metric depends on the number of classes in the dataset, which limits its ability to estimate generalization. To fix this issue, we propose a new metric, which measures retrieval performance, and, unlike Recall@K, estimates generalization. We apply the proposed metric to popular image retrieval methods and provide new insights about deep metric learning generalization.