Advanced Black-Box Tuning of Large Language Models with Limited API Calls

TL;DR

This paper introduces a novel black-box tuning method for large language models that uses a Gaussian Process surrogate to minimize API calls, significantly improving accuracy and efficiency in model adaptation.

Contribution

The paper proposes a new black-box tuning approach that leverages a Gaussian Process surrogate with LogitMap Pairs to reduce API queries while maintaining high accuracy.

Findings

Model accuracy improved from 55.92% to 86.85%.

API query frequency reduced to 1.38%.

Outperforms offline and query-intensive methods.

Abstract

Black-box tuning is an emerging paradigm for adapting large language models (LLMs) to better achieve desired behaviors, particularly when direct access to model parameters is unavailable. Current strategies, however, often present a dilemma of suboptimal extremes: either separately train a small proxy model and then use it to shift the predictions of the foundation model, offering notable efficiency but often yielding limited improvement; or making API calls in each tuning iteration to the foundation model, which entails prohibitive computational costs. Therefore, we propose a novel advanced black-box tuning method for LLMs with limited API calls. Our core strategy involves training a Gaussian Process (GP) surrogate model with "LogitMap Pairs" derived from querying the foundation model on a minimal but highly informative training subset. This surrogate can approximate the outputs of the foundation model to guide the training of the proxy model, thereby effectively reducing the need for direct queries to the foundation model. Extensive experiments verify that our approach elevates pre-trained language model accuracy from 55.92% to 86.85%, reducing the frequency of API queries to merely 1.38%. This significantly outperforms offline approaches that operate entirely without API access. Notably, our method also achieves comparable or superior accuracy to query-intensive approaches, while significantly reducing API costs. This offers a robust and high-efficiency paradigm for language model adaptation.