Efficient Prompt Optimization Through the Lens of Best Arm Identification

TL;DR

This paper introduces TRIPLE, a novel framework for prompt selection in large language models that optimizes the process under explicit budget constraints by leveraging techniques from multi-armed bandit best arm identification.

Contribution

It establishes a new connection between prompt optimization and fixed-budget best arm identification, enabling more efficient prompt selection under cost constraints.

Findings

TRIPLE outperforms baseline methods in various tasks.

It effectively incorporates budget constraints into prompt selection.

Extensions of TRIPLE improve few-shot prompt example selection.

Abstract

The remarkable instruction-following capability of large language models (LLMs) has sparked a growing interest in automatically finding good prompts, i.e., prompt optimization. Most existing works follow the scheme of selecting from a pre-generated pool of candidate prompts. However, these designs mainly focus on the generation strategy, while limited attention has been paid to the selection method. Especially, the cost incurred during the selection (e.g., accessing LLM and evaluating the responses) is rarely explicitly considered. To overcome this limitation, this work provides a principled framework, TRIPLE, to efficiently perform prompt selection under an explicit budget constraint. TRIPLE is built on a novel connection established between prompt optimization and fixed-budget best arm identification (BAI-FB) in multi-armed bandits (MAB); thus, it is capable of leveraging the rich toolbox from BAI-FB systematically and also incorporating unique characteristics of prompt optimization. Extensive experiments on multiple well-adopted tasks using various LLMs demonstrate the remarkable performance improvement of TRIPLE over baselines while satisfying the limited budget constraints. As an extension, variants of TRIPLE are proposed to efficiently select examples for few-shot prompts, also achieving superior empirical performance.