RE-AdaptIR: Improving Information Retrieval through Reverse Engineered Adaptation

TL;DR

This paper introduces RE-AdaptIR, a method that enhances large language model-based information retrieval systems using only unlabeled data, achieving better results across multiple domains without requiring labeled examples.

Contribution

The paper presents a novel approach, RE-AdaptIR, for improving IR models through reverse engineered adaptation using unlabeled data, reducing reliance on costly labeled datasets.

Findings

Improved IR performance in both training and unseen domains.

Effective zero-shot adaptation without labeled data.

Insights into fine-tuning scenarios for IR models.

Abstract

Large language models (LLMs) fine-tuned for text-retrieval have demonstrated state-of-the-art results across several information retrieval (IR) benchmarks. However, supervised training for improving these models requires numerous labeled examples, which are generally unavailable or expensive to acquire. In this work, we explore the effectiveness of extending reverse engineered adaptation to the context of information retrieval (RE-AdaptIR). We use RE-AdaptIR to improve LLM-based IR models using only unlabeled data. We demonstrate improved performance both in training domains as well as zero-shot in domains where the models have seen no queries. We analyze performance changes in various fine-tuning scenarios and offer findings of immediate use to practitioners.