In-Context Black-Box Optimization with Unreliable Feedback

TL;DR

This paper introduces FICBO, a feedback-aware transformer-based optimizer that leverages auxiliary feedback sources to improve black-box optimization, demonstrating robustness and effectiveness across synthetic and real-world tasks.

Contribution

It proposes a structured feedback prior and a pretrained transformer model for feedback-informed in-context optimization, enabling better source reliability estimation and query selection.

Findings

FICBO outperforms baseline methods on synthetic and real-world tasks.

The model effectively exploits informative feedback and remains robust to misleading sources.

Empirical analysis reveals the model's interpretability and decision-making insights.

Abstract

Black-box optimization in science and engineering often comes with side information: experts, simulators, pretrained predictors, or heuristics can suggest which candidates look promising. This information can accelerate search, but it can also be biased, input-dependent, or misleading. Feedback-aware BO methods typically handle one task at a time, limiting their ability to generalize over multiple sources of feedback. In-context optimizers address cross-task adaptation, but usually assume that optimization history is the only available signal at test time. We study feedback-informed in-context black-box optimization (FICBO), where a pretrained optimizer conditions on both the observed history and cheap auxiliary feedback for the current candidate set. We introduce a structured feedback prior that models how feedback sources vary in their access, relevance, and distortion relative to the true objective, and use it to pretrain a feedback-aware transformer. At test time, the model estimates source reliability in context by comparing observed objective values with auxiliary signals, improving query selection. On synthetic and real-world tasks, FICBO effectively exploits informative feedback while remaining robust to weak or misleading sources, improving over other baselines. Empirical investigations further illustrate how the model perceives test-time sources, offering insights into its interpretability and decision-making process.