Near-Optimal Algorithm for Non-Stationary Kernelized Bandits

TL;DR

This paper introduces a near-optimal algorithm for non-stationary kernelized bandit problems, achieving regret bounds close to the theoretical lower bound while significantly reducing computational costs.

Contribution

It presents the first regret lower bound for non-stationary kernelized bandits and proposes a computationally efficient near-optimal algorithm called R-PERP.

Findings

The regret upper bound matches the regret lower bound, indicating near-optimality.

The R-PERP algorithm effectively reduces computational costs compared to existing methods.

A new confidence bound tailored for non-stationary problems enhances algorithm performance.

Abstract

This paper studies a non-stationary kernelized bandit (KB) problem, also called time-varying Bayesian optimization, where one seeks to minimize the regret under an unknown reward function that varies over time. In particular, we focus on a near-optimal algorithm whose regret upper bound matches the regret lower bound. For this goal, we show the first algorithm-independent regret lower bound for non-stationary KB with squared exponential and Mat\'ern kernels, which reveals that an existing optimization-based KB algorithm with slight modification is near-optimal. However, this existing algorithm suffers from feasibility issues due to its huge computational cost. Therefore, we propose a novel near-optimal algorithm called restarting phased elimination with random permutation (R-PERP), which bypasses the huge computational cost. A technical key point is the simple permutation procedures of query candidates, which enable us to derive a novel tighter confidence bound tailored to the non-stationary problems.