ICLERB: In-Context Learning Embedding and Reranker Benchmark

TL;DR

This paper introduces ICLERB, a new benchmark for evaluating retrieval methods in in-context learning, and proposes RLRAIF, a reinforcement learning approach to improve retrieval models based on LLM feedback.

Contribution

The paper presents ICLERB as a novel benchmark for ICL retrieval and introduces RLRAIF, a new training algorithm that enhances retrieval performance using minimal LLM feedback.

Findings

Small models fine-tuned with RLRAIF outperform large state-of-the-art retrieval models.

ICLERB reveals significant differences from existing benchmarks, emphasizing the need for specialized evaluation.

Retrieval as a recommendation problem improves ICL performance.

Abstract

In-Context Learning (ICL) enables Large Language Models (LLMs) to perform new tasks by conditioning on prompts with relevant information. Retrieval-Augmented Generation (RAG) enhances ICL by incorporating retrieved documents into the LLM's context at query time. However, traditional retrieval methods focus on semantic relevance, treating retrieval as a search problem. In this paper, we propose reframing retrieval for ICL as a recommendation problem, aiming to select documents that maximize utility in ICL tasks. We introduce the In-Context Learning Embedding and Reranker Benchmark (ICLERB), a novel evaluation framework that compares retrievers based on their ability to enhance LLM accuracy in ICL settings. Additionally, we propose a novel Reinforcement Learning-to-Rank from AI Feedback (RLRAIF) algorithm, designed to fine-tune retrieval models using minimal feedback from the LLM. Our experimental results reveal notable differences between ICLERB and existing benchmarks, and demonstrate that small models fine-tuned with our RLRAIF algorithm outperform large state-of-the-art retrieval models. These findings highlight the limitations of existing evaluation methods and the need for specialized benchmarks and training strategies adapted to ICL.