Learning Deep Representations via Contrastive Learning for Instance Retrieval

TL;DR

This paper introduces a contrastive learning approach tailored for instance retrieval, demonstrating that fine-tuning CL models with AP loss significantly improves retrieval accuracy on benchmark datasets.

Contribution

It is the first to adapt contrastive learning specifically for instance retrieval, proposing a new training strategy that enhances discriminative features for this task.

Findings

Contrastive learning models outperform pre-trained DNN features in IIR.

Fine-tuning with AP loss improves retrieval performance.

Significant gains on Oxford and Paris datasets.

Abstract

Instance-level Image Retrieval (IIR), or simply Instance Retrieval, deals with the problem of finding all the images within an dataset that contain a query instance (e.g. an object). This paper makes the first attempt that tackles this problem using instance-discrimination based contrastive learning (CL). While CL has shown impressive performance for many computer vision tasks, the similar success has never been found in the field of IIR. In this work, we approach this problem by exploring the capability of deriving discriminative representations from pre-trained and fine-tuned CL models. To begin with, we investigate the efficacy of transfer learning in IIR, by comparing off-the-shelf features learned by a pre-trained deep neural network (DNN) classifier with features learned by a CL model. The findings inspired us to propose a new training strategy that optimizes CL towards learning IIR-oriented features, by using an Average Precision (AP) loss together with a fine-tuning method to learn contrastive feature representations that are tailored to IIR. Our empirical evaluation demonstrates significant performance enhancement over the off-the-shelf features learned from a pre-trained DNN classifier on the challenging Oxford and Paris datasets.