BayMOTH: Bayesian optiMizatiOn with meTa-lookahead -- a simple approacH

TL;DR

BayMOTH introduces a simple meta-BO method that adaptively uses related-task information or lookahead to improve optimization efficiency across diverse tasks.

Contribution

The paper proposes a unified meta-BO algorithm that adaptively leverages related-task information or fallback lookahead, enhancing robustness and efficiency.

Findings

Competitive performance on function optimization tasks.

Effective in low task-relatedness regimes.

Maintains strong performance when task structure transfer is limited.

Abstract

Bayesian optimization (BO) has for sequential optimization of expensive black-box functions demonstrated practicality and effectiveness in many real-world settings. Meta-Bayesian optimization (meta-BO) focuses on improving the sample efficiency of BO by making use of information from related tasks. Although meta-BO is sample-efficient when task structure transfers, poor alignment between meta-training and test tasks can cause suboptimal queries to be suggested during online optimization. To this end, we propose a simple meta-BO algorithm that utilizes related-task information when determined useful, falling back to lookahead otherwise, within a unified framework. We demonstrate competitiveness of our method with existing approaches on function optimization tasks, while retaining strong performance in low task-relatedness regimes where test tasks share limited structure with the meta-training set.