Stochastic Bayesian Optimization with Unknown Continuous Context Distribution via Kernel Density Estimation

TL;DR

This paper introduces two kernel density estimation-based algorithms for Bayesian optimization over unknown continuous context distributions, achieving sub-linear regret and demonstrating effectiveness through experiments.

Contribution

The paper proposes novel algorithms for BO with unknown context distributions, including a distributionally robust approach, with theoretical regret guarantees.

Findings

Both algorithms achieve sub-linear Bayesian cumulative regret.

Numerical experiments confirm the algorithms' effectiveness.

The second algorithm handles complex distributions better.

Abstract

Bayesian optimization (BO) is a sample-efficient method and has been widely used for optimizing expensive black-box functions. Recently, there has been a considerable interest in BO literature in optimizing functions that are affected by context variable in the environment, which is uncontrollable by decision makers. In this paper, we focus on the optimization of functions' expectations over continuous context variable, subject to an unknown distribution. To address this problem, we propose two algorithms that employ kernel density estimation to learn the probability density function (PDF) of continuous context variable online. The first algorithm is simpler, which directly optimizes the expectation under the estimated PDF. Considering that the estimated PDF may have high estimation error when the true distribution is complicated, we further propose the second algorithm that optimizes the distributionally robust objective. Theoretical results demonstrate that both algorithms have sub-linear Bayesian cumulative regret on the expectation objective. Furthermore, we conduct numerical experiments to empirically demonstrate the effectiveness of our algorithms.