A Data-Driven Bayesian Nonparametric Approach for Black-Box Optimization

TL;DR

This paper introduces a Bayesian nonparametric method for optimizing stochastic black-box functions by accounting for distribution estimation errors, with proven convergence and an efficient algorithm demonstrated through experiments.

Contribution

It proposes a novel data-driven Bayesian nonparametric framework for black-box optimization that relaxes parametric assumptions and accounts for finite-data errors.

Findings

The DaBNO objective converges asymptotically to the true objective.

The DaBNO-K algorithm demonstrates empirical global convergence.

Numerical results show strong finite-sample performance.

Abstract

We present a data-driven Bayesian nonparametric approach for global optimization (DaBNO) of stochastic black-box function. The function value depends on the distribution of a random vector. However, this distribution is usually complex and hardly known in practice, and is often inferred from data (realizations of random vectors). The DaBNO accounts for the finite-data error that arises when estimating the distribution and relaxes the commonly-used parametric assumption to reduce the distribution-misspecified error. We show that the DaBNO objective formulation can converge to the true objective asymptotically. We further develop a surrogate-assisted algorithm DaBNO-K to efficiently optimize the proposed objective function based on a carefully designed kernel. Numerical experiments are conducted with several synthetic and practical problems, demonstrating the empirical global convergence of this algorithm and its finite-sample performance.