Bayesian task embedding for few-shot Bayesian optimization

TL;DR

This paper introduces a Bayesian task embedding approach that leverages data from multiple systems with unknown relationships to improve Bayesian optimization, especially in low-data scenarios, using a unified probabilistic metamodel.

Contribution

It presents a novel method that incorporates multiple systems into a single Bayesian optimization framework via latent variables, enabling effective optimization with limited data.

Findings

Outperforms traditional Bayesian optimization in few-shot settings

Effective on synthetic and real-world examples

Reduces data requirements for system-specific optimization

Abstract

We describe a method for Bayesian optimization by which one may incorporate data from multiple systems whose quantitative interrelationships are unknown a priori. All general (nonreal-valued) features of the systems are associated with continuous latent variables that enter as inputs into a single metamodel that simultaneously learns the response surfaces of all of the systems. Bayesian inference is used to determine appropriate beliefs regarding the latent variables. We explain how the resulting probabilistic metamodel may be used for Bayesian optimization tasks and demonstrate its implementation on a variety of synthetic and real-world examples, comparing its performance under zero-, one-, and few-shot settings against traditional Bayesian optimization, which usually requires substantially more data from the system of interest.