Gaussian Process Upper Confidence Bound Achieves Nearly-Optimal Regret in Noise-Free Gaussian Process Bandits

TL;DR

This paper proves that the GP-UCB algorithm achieves nearly optimal regret bounds in noise-free Gaussian process bandit problems, matching its strong empirical performance with theoretical guarantees.

Contribution

It establishes the first nearly optimal regret bounds for noise-free GP-UCB, including constant cumulative regret for common kernels.

Findings

GP-UCB attains nearly optimal regret bounds in noise-free settings.

First theoretical proof of constant cumulative regret for squared exponential and Matérn kernels.

Bridges the gap between empirical success and theoretical analysis of GP-UCB.

Abstract

We study the noise-free Gaussian Process (GP) bandits problem, in which the learner seeks to minimize regret through noise-free observations of the black-box objective function lying on the known reproducing kernel Hilbert space (RKHS). Gaussian process upper confidence bound (GP-UCB) is the well-known GP-bandits algorithm whose query points are adaptively chosen based on the GP-based upper confidence bound score. Although several existing works have reported the practical success of GP-UCB, the current theoretical results indicate its suboptimal performance. However, GP-UCB tends to perform well empirically compared with other nearly optimal noise-free algorithms that rely on a non-adaptive sampling scheme of query points. This paper resolves this gap between theoretical and empirical performance by showing the nearly optimal regret upper bound of noise-free GP-UCB. Specifically, our analysis shows the first constant cumulative regret in the noise-free settings for the squared exponential kernel and Mat\'ern kernel with some degree of smoothness.