Discrete Prompt Optimization via Constrained Generation for Zero-shot Re-ranker

TL;DR

This paper introduces Co-Prompt, a discrete prompt optimization method that enhances zero-shot re-ranking performance in information retrieval by guiding prompt generation without model fine-tuning, resulting in more effective and interpretable prompts.

Contribution

The paper presents a novel discrete prompt optimization technique, Co-Prompt, which improves zero-shot re-ranking by guiding prompt generation based on a metric, without requiring parameter updates.

Findings

Co-Prompt outperforms baseline re-ranking methods.

Generated prompts are more interpretable for humans.

The method enhances zero-shot re-ranking performance.

Abstract

Re-rankers, which order retrieved documents with respect to the relevance score on the given query, have gained attention for the information retrieval (IR) task. Rather than fine-tuning the pre-trained language model (PLM), the large-scale language model (LLM) is utilized as a zero-shot re-ranker with excellent results. While LLM is highly dependent on the prompts, the impact and the optimization of the prompts for the zero-shot re-ranker are not explored yet. Along with highlighting the impact of optimization on the zero-shot re-ranker, we propose a novel discrete prompt optimization method, Constrained Prompt generation (Co-Prompt), with the metric estimating the optimum for re-ranking. Co-Prompt guides the generated texts from PLM toward optimal prompts based on the metric without parameter update. The experimental results demonstrate that Co-Prompt leads to outstanding re-ranking performance against the baselines. Also, Co-Prompt generates more interpretable prompts for humans against other prompt optimization methods.