Lower Bounds for Time-Varying Kernelized Bandits

TL;DR

This paper establishes the first algorithm-independent lower bounds for non-stationary kernelized bandit problems, highlighting the challenges and gaps in understanding time-varying black-box optimization.

Contribution

It introduces the first lower bounds for non-stationary kernelized bandits under total variation constraints, extending the theoretical understanding beyond stationary cases.

Findings

Lower bounds close to existing upper bounds under $\, ext{ extit{l}}_{ ext{ extit{infty}}}$-norm variations.

Bounds under RKHS norm variations are close but leave open questions about potential improvements.

Highlights the gap between upper and lower bounds in non-stationary kernelized bandits.

Abstract

The optimization of black-box functions with noisy observations is a fundamental problem with widespread applications, and has been widely studied under the assumption that the function lies in a reproducing kernel Hilbert space (RKHS). This problem has been studied extensively in the stationary setting, and near-optimal regret bounds are known via developments in both upper and lower bounds. In this paper, we consider non-stationary scenarios, which are crucial for certain applications but are currently less well-understood. Specifically, we provide the first algorithm-independent lower bounds, where the time variations are subject satisfying a total variation budget according to some function norm. Under $\ell_{\infty}$-norm variations, our bounds are found to be close to an existing upper bound (Hong et al., 2023). Under RKHS norm variations, the upper and lower bounds are still reasonably close but with more of a gap, raising the interesting open question of whether non-minor improvements in the upper bound are possible.