Accelerated Bayesian Optimization throughWeight-Prior Tuning

TL;DR

This paper proposes a method to accelerate Bayesian optimization by leveraging auxiliary data to learn a more suitable weight prior for Gaussian Process models, improving efficiency in practical applications.

Contribution

It introduces a technique to incorporate auxiliary data into the weight prior of Gaussian Processes, enhancing Bayesian optimization performance.

Findings

Accelerated BO on test functions.

Improved BO in polymer fibre manufacturing.

Effective use of auxiliary data for prior learning.

Abstract

Bayesian optimization (BO) is a widely-used method for optimizing expensive (to evaluate) problems. At the core of most BO methods is the modeling of the objective function using a Gaussian Process (GP) whose covariance is selected from a set of standard covariance functions. From a weight-space view, this models the objective as a linear function in a feature space implied by the given covariance K, with an arbitrary Gaussian weight prior ${\bf w} \sim \mathcal{N} ({\bf 0}, {\bf I})$. In many practical applications there is data available that has a similar (covariance) structure to the objective, but which, having different form, cannot be used directly in standard transfer learning. In this paper we show how such auxiliary data may be used to construct a GP covariance corresponding to a more appropriate weight prior for the objective function. Building on this, we show that we may accelerate BO by modeling the objective function using this (learned) weight prior, which we demonstrate on both test functions and a practical application to short-polymer fibre manufacture.