MUMBO: MUlti-task Max-value Bayesian Optimization

TL;DR

MUMBO introduces a scalable, efficient multi-task Bayesian optimization method that significantly reduces computational costs while maintaining high performance, enabling practical applications in complex, multi-fidelity problems.

Contribution

This paper presents MUMBO, the first high-performing, computationally efficient multi-task Bayesian optimization acquisition function based on a novel multi-task entropy search.

Findings

MUMBO achieves robust performance across various optimization tasks.

It significantly reduces computational overhead compared to existing methods.

MUMBO scales well with complex parameter and fidelity spaces.

Abstract

We propose MUMBO, the first high-performing yet computationally efficient acquisition function for multi-task Bayesian optimization. Here, the challenge is to perform efficient optimization by evaluating low-cost functions somehow related to our true target function. This is a broad class of problems including the popular task of multi-fidelity optimization. However, while information-theoretic acquisition functions are known to provide state-of-the-art Bayesian optimization, existing implementations for multi-task scenarios have prohibitive computational requirements. Previous acquisition functions have therefore been suitable only for problems with both low-dimensional parameter spaces and function query costs sufficiently large to overshadow very significant optimization overheads. In this work, we derive a novel multi-task version of entropy search, delivering robust performance with low computational overheads across classic optimization challenges and multi-task hyper-parameter tuning. MUMBO is scalable and efficient, allowing multi-task Bayesian optimization to be deployed in problems with rich parameter and fidelity spaces.