Time-varying Gaussian Process Bandit Optimization with Experts: no-regret in logarithmically-many side queries

TL;DR

This paper introduces a Gaussian Process bandit optimization method that achieves no-regret in a time-varying setting by using only logarithmically-many side queries to an expert, effectively managing temporal changes.

Contribution

It develops a novel approach combining heteroscedastic Gaussian Process regression and sparse inference to attain no-regret with minimal expert queries in dynamic environments.

Findings

Achieves no-regret with logarithmically-many side queries per step.

Effectively manages temporal drift in time-varying optimization.

Extends sparse inference techniques to heteroscedastic Gaussian Processes.

Abstract

We study a time-varying Bayesian optimization problem with bandit feedback, where the reward function belongs to a Reproducing Kernel Hilbert Space (RKHS). We approach the problem via an upper-confidence bound Gaussian Process algorithm, which has been proven to yield no-regret in the stationary case. The time-varying case is more challenging and no-regret results are out of reach in general in the standard setting. As such, we instead tackle the question of how many additional observations asked to an expert are required to regain a no-regret property. To do so, we formulate the presence of past observation via an uncertainty injection procedure, and we reframe the problem as a heteroscedastic Gaussian Process regression. In addition, to achieve a no-regret result, we discard long outdated observations and replace them with updated (possibly very noisy) ones obtained by asking queries to an external expert. By leveraging and extending sparse inference to the heteroscedastic case, we are able to secure a no-regret result in a challenging time-varying setting with only logarithmically-many side queries per time step. Our method demonstrates that minimal additional information suffices to counteract temporal drift, ensuring efficient optimization despite time variation.