Test-time Training for Data-efficient UCDR

TL;DR

This paper introduces a test-time training approach for universal cross-domain retrieval that adapts pre-trained models on test data using self-supervised learning, improving data efficiency in generalized retrieval scenarios.

Contribution

It pioneers the use of test-time adaptation with self-supervised learning for data-efficient universal cross-domain retrieval, without relying on multi-domain training data.

Findings

Self-supervised loss functions like RotNet, Jigsaw, Barlow Twins are effective.

The proposed method is simple, easy to implement, and improves retrieval performance.

Extensive experiments validate the approach's effectiveness in data-efficient UCDR.

Abstract

Image retrieval under generalized test scenarios has gained significant momentum in literature, and the recently proposed protocol of Universal Cross-domain Retrieval is a pioneer in this direction. A common practice in any such generalized classification or retrieval algorithm is to exploit samples from many domains during training to learn a domain-invariant representation of data. Such criterion is often restrictive, and thus in this work, for the first time, we explore the generalized retrieval problem in a data-efficient manner. Specifically, we aim to generalize any pre-trained cross-domain retrieval network towards any unknown query domain/category, by means of adapting the model on the test data leveraging self-supervised learning techniques. Toward that goal, we explored different self-supervised loss functions~(for example, RotNet, JigSaw, Barlow Twins, etc.) and analyze their effectiveness for the same. Extensive experiments demonstrate the proposed approach is simple, easy to implement, and effective in handling data-efficient UCDR.