Distributionally Robust Bayesian Quadrature Optimization

TL;DR

This paper introduces a distributionally robust Bayesian quadrature optimization method that accounts for uncertainty in the probability distribution, using a novel posterior sampling algorithm to improve robustness and convergence.

Contribution

It proposes the DRBQO algorithm that maximizes the objective under worst-case distributional scenarios, addressing limitations of standard BQO with limited samples.

Findings

Empirically effective in synthetic and real-world problems

Theoretically characterized by Bayesian regret convergence

Outperforms non-robust methods in uncertain distribution settings

Abstract

Bayesian quadrature optimization (BQO) maximizes the expectation of an expensive black-box integrand taken over a known probability distribution. In this work, we study BQO under distributional uncertainty in which the underlying probability distribution is unknown except for a limited set of its i.i.d. samples. A standard BQO approach maximizes the Monte Carlo estimate of the true expected objective given the fixed sample set. Though Monte Carlo estimate is unbiased, it has high variance given a small set of samples; thus can result in a spurious objective function. We adopt the distributionally robust optimization perspective to this problem by maximizing the expected objective under the most adversarial distribution. In particular, we propose a novel posterior sampling based algorithm, namely distributionally robust BQO (DRBQO) for this purpose. We demonstrate the empirical effectiveness of our proposed framework in synthetic and real-world problems, and characterize its theoretical convergence via Bayesian regret.