The Vizier Gaussian Process Bandit Algorithm

TL;DR

This paper details the implementation and design of Google Vizier's Gaussian Process Bandit Algorithm, highlighting its robustness and versatility through benchmark experiments and emphasizing its evolution via research and user feedback.

Contribution

It presents the current default algorithm of Open Source Vizier, showcasing improvements and practical performance in large-scale Bayesian optimization.

Findings

Robustness against industry benchmarks

Versatility across practical modes

Effective large-scale Bayesian optimization

Abstract

Google Vizier has performed millions of optimizations and accelerated numerous research and production systems at Google, demonstrating the success of Bayesian optimization as a large-scale service. Over multiple years, its algorithm has been improved considerably, through the collective experiences of numerous research efforts and user feedback. In this technical report, we discuss the implementation details and design choices of the current default algorithm provided by Open Source Vizier. Our experiments on standardized benchmarks reveal its robustness and versatility against well-established industry baselines on multiple practical modes.