Offline Stochastic Optimization of Black-Box Objective Functions

TL;DR

This paper introduces SOBBO, a novel approach for offline stochastic black-box optimization that effectively handles uncertainties and improves data efficiency through surrogate modeling and gradient estimation techniques.

Contribution

The paper presents SOBBO, a new framework for offline stochastic black-box optimization that addresses uncertainties and enhances robustness and efficiency in data-scarce and data-rich regimes.

Findings

SOBBO outperforms existing methods on synthetic benchmarks.

The surrogate-based approach enables efficient gradient-based optimization.

Gradient estimation under constraints improves robustness in scarce-data scenarios.

Abstract

Many challenges in science and engineering, such as drug discovery and communication network design, involve optimizing complex and expensive black-box functions across vast search spaces. Thus, it is essential to leverage existing data to avoid costly active queries of these black-box functions. To this end, while Offline Black-Box Optimization (BBO) is effective for deterministic problems, it may fall short in capturing the stochasticity of real-world scenarios. To address this, we introduce Stochastic Offline BBO (SOBBO), which tackles both black-box objectives and uncontrolled uncertainties. We propose two solutions: for large-data regimes, a differentiable surrogate allows for gradient-based optimization, while for scarce-data regimes, we directly estimate gradients under conservative field constraints, improving robustness, convergence, and data efficiency. Numerical experiments demonstrate the effectiveness of our approach on both synthetic and real-world tasks.