MALIBO: Meta-learning for Likelihood-free Bayesian Optimization

TL;DR

MALIBO introduces a meta-learning approach for Bayesian optimization that directly learns query utility, models task uncertainty, and enhances robustness, outperforming existing methods in diverse benchmarks.

Contribution

The paper proposes a surrogate-free meta-learning Bayesian optimization method that explicitly models task uncertainty and improves robustness to task variations.

Findings

Outperforms state-of-the-art meta-learning BO methods in benchmarks.

Demonstrates strong anytime performance across tasks.

Effectively handles limited observations and task differences.

Abstract

Bayesian optimization (BO) is a popular method to optimize costly black-box functions. While traditional BO optimizes each new target task from scratch, meta-learning has emerged as a way to leverage knowledge from related tasks to optimize new tasks faster. However, existing meta-learning BO methods rely on surrogate models that suffer from scalability issues and are sensitive to observations with different scales and noise types across tasks. Moreover, they often overlook the uncertainty associated with task similarity. This leads to unreliable task adaptation when only limited observations are obtained or when the new tasks differ significantly from the related tasks. To address these limitations, we propose a novel meta-learning BO approach that bypasses the surrogate model and directly learns the utility of queries across tasks. Our method explicitly models task uncertainty and includes an auxiliary model to enable robust adaptation to new tasks. Extensive experiments show that our method demonstrates strong anytime performance and outperforms state-of-the-art meta-learning BO methods in various benchmarks.