Class Anchor Margin Loss for Content-Based Image Retrieval

TL;DR

This paper introduces a novel loss function for content-based image retrieval that directly optimizes feature separation without pair mining, improving retrieval accuracy and efficiency across multiple datasets.

Contribution

The paper proposes a new repeller-attractor loss that directly optimizes features for CBIR, eliminating pair mining and enhancing retrieval performance with a two-stage system.

Findings

Outperforms existing loss functions on multiple datasets

Enables efficient two-stage retrieval without exhaustive comparisons

Works effectively with both CNN and transformer architectures

Abstract

The performance of neural networks in content-based image retrieval (CBIR) is highly influenced by the chosen loss (objective) function. The majority of objective functions for neural models can be divided into metric learning and statistical learning. Metric learning approaches require a pair mining strategy that often lacks efficiency, while statistical learning approaches are not generating highly compact features due to their indirect feature optimization. To this end, we propose a novel repeller-attractor loss that falls in the metric learning paradigm, yet directly optimizes for the L2 metric without the need of generating pairs. Our loss is formed of three components. One leading objective ensures that the learned features are attracted to each designated learnable class anchor. The second loss component regulates the anchors and forces them to be separable by a margin, while the third objective ensures that the anchors do not collapse to zero. Furthermore, we develop a more efficient two-stage retrieval system by harnessing the learned class anchors during the first stage of the retrieval process, eliminating the need of comparing the query with every image in the database. We establish a set of four datasets (CIFAR-100, Food-101, SVHN, and Tiny ImageNet) and evaluate the proposed objective in the context of few-shot and full-set training on the CBIR task, by using both convolutional and transformer architectures. Compared to existing objective functions, our empirical evidence shows that the proposed objective is generating superior and more consistent results.