Scenario Approach for Robust Blackbox Optimization in the Bandit Setting

TL;DR

This paper introduces a scenario-based approach for robust blackbox optimization in bandit settings, leveraging Gaussian Process models and UCB strategies to bound and analyze regret under uncertain parameters.

Contribution

It presents a novel iterative algorithm using scenario sampling and UCB for robust blackbox optimization, with theoretical regret bounds and asymptotic analysis.

Findings

High probability upper bounds on regret under re-draw.

Conditions for regret to asymptotically tend to zero.

Numerical results demonstrating algorithm performance.

Abstract

This paper discusses a scenario approach to robust optimization of a blackbox function in a bandit setting. We assume that the blackbox function can be modeled as a Gaussian Process (GP) for every realization of the uncertain parameter. We adopt a scenario approach in which we draw fixed independent samples of the uncertain parameter. For a given policy, i.e., a sequence of query points and uncertain parameters in the sampled set, we introduce a notion of regret defined with respect to additional draws of the uncertain parameter, termed as scenario regret under re-draw. We present a scenario-based iterative algorithm using the upper confidence bound (UCB) of the fixed independent scenarios to compute a policy for the blackbox optimization. For this algorithm, we characterize a high probability upper bound on the regret under re-draw for any finite number of iterations of the algorithm. We further characterize parameter regimes in which the regret tends to zero asymptotically with the number of iterations with high probability. Finally, we supplement our analysis with numerical results.