Gaussian Process Thompson Sampling via Rootfinding

TL;DR

This paper introduces a novel global optimization method for Gaussian Process Thompson Sampling that efficiently finds all local optima, significantly enhancing Bayesian optimization performance, especially in high-dimensional spaces.

Contribution

It proposes a rootfinding-based approach to global optimization in GP-TS, enabling better local optima identification and improved Bayesian optimization results.

Findings

Remarkable improvement in global optimization of GP samples

Outperforms GP-UCB and EI in Bayesian optimization tasks

Effective in high-dimensional optimization problems

Abstract

Thompson sampling (TS) is a simple, effective stochastic policy in Bayesian decision making. It samples the posterior belief about the reward profile and optimizes the sample to obtain a candidate decision. In continuous optimization, the posterior of the objective function is often a Gaussian process (GP), whose sample paths have numerous local optima, making their global optimization challenging. In this work, we introduce an efficient global optimization strategy for GP-TS that carefully selects starting points for gradient-based multi-start optimizers. It identifies all local optima of the prior sample via univariate global rootfinding, and optimizes the posterior sample using a differentiable, decoupled representation. We demonstrate remarkable improvement in the global optimization of GP posterior samples, especially in high dimensions. This leads to dramatic improvements in the overall performance of Bayesian optimization using GP-TS acquisition functions, surprisingly outperforming alternatives like GP-UCB and EI.